Monday, 18 December 2017

Power Meter vs Smart Trainer Power - Hello Cadence. Goodbye Accuracy? (ERG Mode)

Here's a brief run-down of a day of testing trying to get to the bottom of why I'm seeing some major differences in power numbers (well beyond the maximum combined margin of error) when putting my Garmin Vector 3 Power Meter pedals up against the Tacx Neo Smart Trainer in steady state ERG mode.


After receiving my Vector 3 pedals (retail/production) last week, I installed them as per the manual, went 45km ride outside on them (sprints, etc to bed them in, then performed my standard Lama Lab Test on them with the Tacx Neo Smart Trainer (control/baseline for power and ERG). The full ~16 minute video is over on YouTube

Aside from the no-show for Bluetooth power meter support (at this stage) the other glaring issue was the +20W discrepancy between the Tacx Neo and the Vector 3 at 200-220W during a sustained ERG effort. This discrepancy is greater than the combined accuracy error margins (Neo -+1%, Vector -+1%, and a generous drivetrain loss estimation of -+5%). What was strange is this difference wasn't seen when 'just riding along' or in a continual ramp test I performed in SIM mode. So it wasn't going to be a simple calibration/zero-offset fix.

Today's Tests:

Cadence. What was the impact of changing the pedalling cadence on the ERG mode power readings, using the same gear? I typically pedal at 90-95rpm for my ERG mode tests and haven't seen any major power reporting issues in those zones... until now. 

Test 1 - Tacx Neo | Garmin Vector 3

- Switching from 150W ERG to 200W ERG at the 12:30 minute mark
- The higher the cadence, the more the Neo and Vector 3 disagree on power. 
- Cadence under 80rpm shows acceptable accuracy ranges. 
- Cadence greater than 100rpm results in power reporting up to +20W from the Vector 3. 

Test 2 - Tacx Neo | PowerTap P1

- ~+10-12W differences in power reported at higher cadence ~110 (Vector 3 was ~+20W). 
- Lower cadences ~60rpm appear optimal for having the Neo and P1 agree on power. 

Test 3 - Discarded (Forgot to perform a sensor calibration on the Elite Drivo)

Test 4 - Elite Drivo | Garmin Vector 3

- ~+10W differences in power reported at 90rpm+ 
- Lower cadences ~60rpm appear optimal for having the Drivo and Vector 3 agree on power. 

Test 5 - Elite Drivo | PowerTap P1

- NICE!.... Only start to see disagreement at 130rpm+.


ERG is a strange beast. It's never a constant application of resistance from the Smart Trainer. It'll oscillate slightly above and slightly below the set target watts as the very inefficient human motor stomps away on the pedals. This means the power measurement becomes a complex task.

At this point, I'll conclude: 

- The power reporting of the current Vector 3 pedals (Firmware 2.30.0) appears to be influenced greatly by cadence when in steady state ERG mode on an indoor trainer. Lower cadences appear to match smart trainers with power accuracy ranges of -+1%.

- The PowerTap P1 pedals appear to also suffer from high rpm power reporting differences, but at much higher RPM than the Vector 3 and with a smaller margin of error.


Could this be related to sample rate at which these power meters calculate power? 

Side thought..... The PowerTap P1 are compatible with oval chain-rings, which means they're doing some high sample-rate and averaging due to the changing velocity of the pedal stroke. Is this why they have the edge over the Vector 3 (no support for oval rings) for these higher RPM ERG tests?

My focus here is on understanding what's happening with the Vector 3 pedals, I also have a suspicion the Tacx Neo itself starts to struggle with power reporting at higher RPM ERG efforts. I did have a Quarq power meter on for a number of these tests..... but I've omitted that for now as I focus purely on the pedals vs the world. 

As always... more data needed. Stay tuned.

Next day updates...

Because this is 100% doing my head in.

Static weight tests/verification on the Vector 3:

20.000kg certified weight + 21g hook/mount. 172.5mm cranks. Calculated Nm: 33.845 

Readings from the Edge 520:

R: 33.41, 33.50, 33.47. 33.46 avg. 98.86% of calculated Nm.
L: 33.66, 33.59, 33.72. 33.65 avg. 99.42% of calculated Nm.

Test 9 - Vector 3 | Tacx Neo. 

Post static weight tests on Vector3

39/15 gearing. No gear changing. 
Head units: Garmin 520, Garmin 820. 1 second recording. Latest firmware on both.

- 3 minutes 200W ERG test. Cadence change every minute. (90/60/120)
- Cycling Dynamics turned off at 5min.
- 3 minutes 200W ERG tests / cadence change every minute. (90/60/120)
- Cadence Ramp Test ERG enabled (Cadence increase, power 'should' remain the same) 
- Cadence Ramp Test ERG disabled (Cadence increase, power increase)


Test 10 - PowerTap P1 | Tacx Neo

Everything IDENTICAL to Test 9, except for the pedals. I skipped the 2nd 3minute block I used in Test 9 after disabling Cycling Dynamics (showed no change)


The raw data files should be accessible on Ray's site for download / further review. 

In summary and for clarification - I really want the Vector 3 to work, just as well as the P1, it not better. I won't rule out the Neo, my testing protocol, my wonkyness at higher RPM on L/R, maybe the moon phase.... I just can't explain or resolve what I'm seeing with the readings. 

Grabbing at straws...

- Installation torque? I'm snugging them on just as tight as the P1.
- Interface between the pedal and the cranks? I've tried with and without the 2mm washers, no change on the Ultegra 6800 cranks. 
- Cleats?.... I'm running 6deg float on V3, 0deg on the P1. I have the 0deg Exustar ones here in a box I'm yet to install. 
- Sample rates.... well beyond what I've got access to see/review/know about. I'll leave that for the engineers. 

Stay tuned..... again! 


Liamo said...

Shoes? If the shoes with the Vector cleats are more flexible than those with the P1 cleats it might be a contributor?

I'd still expect to see those differences flow through to the trainer though..

Unknown said...

I get the exact same issue with the Vector 2s as well. Difference if 20W at above 200W and 95 cadence but that is even without ERG mode.

Harry said...

I'd like to see the data files for Sim (Zwift I assume) on both the Neo and Drivo using both the V3 and P1.

Am I just missing those files? Can you provide those 4 links?

Any updates coming in the near future?

Shane Miller said...

Zwift is just another head unit, so it's 1:1 data the same as the other head units.

Still waiting for the BLE firmware for these pedals before I reassess the readings. They're going well out on the road, so that's nice.

Pietro Sbisà said...

Great review Shane!
Have you tried the Vector 3 with a Wahoo Elemnt? Does the head unit need to be a Garmin one?

Shane Miller said...

They work with the Elemnt/BOLT minus withe pedalling metrics/dynamics the Edge units have.... for now.