Background:
After receiving my Vector 3 pedals (retail/production) last week, I installed them as per the manual, went 45km ride outside on them (sprints, etc to bed them in, then performed my standard Lama Lab Test on them with the Tacx Neo Smart Trainer (control/baseline for power and ERG). The full ~16 minute video is over on YouTube.
Aside from the no-show for Bluetooth power meter support (at this stage) the other glaring issue was the +20W discrepancy between the Tacx Neo and the Vector 3 at 200-220W during a sustained ERG effort. This discrepancy is greater than the combined accuracy error margins (Neo -+1%, Vector -+1%, and a generous drivetrain loss estimation of -+5%). What was strange is this difference wasn't seen when 'just riding along' or in a continual ramp test I performed in SIM mode. So it wasn't going to be a simple calibration/zero-offset fix.
Today's Tests:
Cadence. What was the impact of changing the pedalling cadence on the ERG mode power readings, using the same gear? I typically pedal at 90-95rpm for my ERG mode tests and haven't seen any major power reporting issues in those zones... until now.
Test 1 - Tacx Neo | Garmin Vector 3
Observations:
- Switching from 150W ERG to 200W ERG at the 12:30 minute mark
- The higher the cadence, the more the Neo and Vector 3 disagree on power.
- Cadence under 80rpm shows acceptable accuracy ranges.
- Cadence greater than 100rpm results in power reporting up to +20W from the Vector 3.
Test 2 - Tacx Neo | PowerTap P1
Observations:
- ~+10-12W differences in power reported at higher cadence ~110 (Vector 3 was ~+20W).
- Lower cadences ~60rpm appear optimal for having the Neo and P1 agree on power.
Test 3 - Discarded (Forgot to perform a sensor calibration on the Elite Drivo)
Test 4 - Elite Drivo | Garmin Vector 3
Observations:
- ~+10W differences in power reported at 90rpm+
- Lower cadences ~60rpm appear optimal for having the Drivo and Vector 3 agree on power.
Test 5 - Elite Drivo | PowerTap P1
Observations:
- NICE!.... Only start to see disagreement at 130rpm+.
Conclusions....
ERG is a strange beast. It's never a constant application of resistance from the Smart Trainer. It'll oscillate slightly above and slightly below the set target watts as the very inefficient human motor stomps away on the pedals. This means the power measurement becomes a complex task.
At this point, I'll conclude:
- The power reporting of the current Vector 3 pedals (Firmware 2.30.0) appears to be influenced greatly by cadence when in steady state ERG mode on an indoor trainer. Lower cadences appear to match smart trainers with power accuracy ranges of -+1%.
- The PowerTap P1 pedals appear to also suffer from high rpm power reporting differences, but at much higher RPM than the Vector 3 and with a smaller margin of error.
Thoughts....
Could this be related to sample rate at which these power meters calculate power?
Side thought..... The PowerTap P1 are compatible with oval chain-rings, which means they're doing some high sample-rate and averaging due to the changing velocity of the pedal stroke. Is this why they have the edge over the Vector 3 (no support for oval rings) for these higher RPM ERG tests?
My focus here is on understanding what's happening with the Vector 3 pedals, I also have a suspicion the Tacx Neo itself starts to struggle with power reporting at higher RPM ERG efforts. I did have a Quarq power meter on for a number of these tests..... but I've omitted that for now as I focus purely on the pedals vs the world.
As always... more data needed. Stay tuned.
Next day updates...
Because this is 100% doing my head in.
Static weight tests/verification on the Vector 3:
20.000kg certified weight + 21g hook/mount. 172.5mm cranks. Calculated Nm: 33.845
Readings from the Edge 520:
R: 33.41, 33.50, 33.47. 33.46 avg. 98.86% of calculated Nm.
L: 33.66, 33.59, 33.72. 33.65 avg. 99.42% of calculated Nm.
Readings from the Edge 520:
R: 33.41, 33.50, 33.47. 33.46 avg. 98.86% of calculated Nm.
L: 33.66, 33.59, 33.72. 33.65 avg. 99.42% of calculated Nm.
Test 9 - Vector 3 | Tacx Neo.
Post static weight tests on Vector3
Post static weight tests on Vector3
39/15 gearing. No gear changing.
Head units: Garmin 520, Garmin 820. 1 second recording. Latest firmware on both.
- 3 minutes 200W ERG test. Cadence change every minute. (90/60/120)
- Cycling Dynamics turned off at 5min.
- 3 minutes 200W ERG tests / cadence change every minute. (90/60/120)
- Cadence Ramp Test ERG enabled (Cadence increase, power 'should' remain the same)
- Cadence Ramp Test ERG disabled (Cadence increase, power increase)
Head units: Garmin 520, Garmin 820. 1 second recording. Latest firmware on both.
- 3 minutes 200W ERG test. Cadence change every minute. (90/60/120)
- Cycling Dynamics turned off at 5min.
- 3 minutes 200W ERG tests / cadence change every minute. (90/60/120)
- Cadence Ramp Test ERG enabled (Cadence increase, power 'should' remain the same)
- Cadence Ramp Test ERG disabled (Cadence increase, power increase)
Data: https://analyze.dcrainmaker.com/#/pu...9-ffdaaa8cd18c
Test 10 - PowerTap P1 | Tacx Neo
Everything IDENTICAL to Test 9, except for the pedals. I skipped the 2nd 3minute block I used in Test 9 after disabling Cycling Dynamics (showed no change)
Data: https://analyze.dcrainmaker.com/#/pu...0-abf3212b6abe
The raw data files should be accessible on Ray's site for download / further review.
In summary and for clarification - I really want the Vector 3 to work, just as well as the P1, it not better. I won't rule out the Neo, my testing protocol, my wonkyness at higher RPM on L/R, maybe the moon phase.... I just can't explain or resolve what I'm seeing with the readings.
Grabbing at straws...
- Installation torque? I'm snugging them on just as tight as the P1.
- Interface between the pedal and the cranks? I've tried with and without the 2mm washers, no change on the Ultegra 6800 cranks.
- Cleats?.... I'm running 6deg float on V3, 0deg on the P1. I have the 0deg Exustar ones here in a box I'm yet to install.
- Sample rates.... well beyond what I've got access to see/review/know about. I'll leave that for the engineers.
Stay tuned..... again!
28 comments:
Shoes? If the shoes with the Vector cleats are more flexible than those with the P1 cleats it might be a contributor?
I'd still expect to see those differences flow through to the trainer though..
I get the exact same issue with the Vector 2s as well. Difference if 20W at above 200W and 95 cadence but that is even without ERG mode.
I'd like to see the data files for Sim (Zwift I assume) on both the Neo and Drivo using both the V3 and P1.
Am I just missing those files? Can you provide those 4 links?
Any updates coming in the near future?
Zwift is just another head unit, so it's 1:1 data the same as the other head units.
Still waiting for the BLE firmware for these pedals before I reassess the readings. They're going well out on the road, so that's nice.
Great review Shane!
Have you tried the Vector 3 with a Wahoo Elemnt? Does the head unit need to be a Garmin one?
Cheers
Pietro
They work with the Elemnt/BOLT minus withe pedalling metrics/dynamics the Edge units have.... for now.
I received the vectors today. Did a Trainerroad workout on my kckr in erg mode with vectors tuned into Garmin 520 head unit. Seeing similar differences to you. At the worst point at a cadance of 82, kickr measured 289 and vectors 354!
Hi!
Thanks for a GREAT article! Having bought a pair of Favero Assioma Duo just two weeks ago I couldn't wait to get on my trainer (the weather and darkness in Sweden during Feb do not allow for outdoor training, if you ask me...). So I have so far only used the pedals indoors together with my trainer Tacx Genius Smart. Here is my headache.
I get the same frustrating results. I get a difference of 20-50 W depending on cadence and power put into the workout. This is really frustrating and I cannot seem to get my head around the reason.
My setup is that I perform according to a predefined workout based on power in my iPad and the Tacx app. The predefined workout makes me get a power measurement as expected independent on cadence. If I increase cadence the Tacx app releases on resistance so that I keep my self in the “right” power area. I get a nice graph over my workout where I see the Tacx power measurements.
My Assioma Duo are connected to my Garmin Edge 1030. They constantly shows a huge difference in power measurements, and we are talking about 20-50 Watt BELOW what the Tacx app is saying.
This behavior is exactly the same if I switch from the Tacx app on my iPad to instead use Zwift for joyriding or performing a workout.
Can you explain this difference and do you have any cure for this? According to an article by DC Rainmaker (https://www.dcrainmaker.com/2017/09/favero-assioma-in-depth-review.html/comment-page-1#comment-2842584) the Assioma Duo have no issues what so ever with being accurate compared to other power meters (including Garmin Vector 3), disregarding if they are indoor trainers or other pedal meters.
Any new knowledge on how to fight this? There is an option of using the Assioma app to correct the powermeter measurements, but how do I know that the Tacx is giving the "correct" value, and not the Assiomas? Perhaps it is the Tacx Genius Smart trainer that is giving the wrong power measurement?
Kind regards
Michael
Hi Shane. Wondering if you have got to the bottom of this. Left a comment on DC Ray's review on the Vector 3's about this and if he had experienced similar and said that he hadn't been able to reproduce this. Wondering if you have sussed out what the issued you saw are related to, whether Garmin can explain this, or whether this issue has disappeared? The whole thing is a little worrying, especially when other people notice similar issues (ref Neil Owens above).
Cheers!
I've been waiting for the BLE firmware to revisit this myself... I just need the time now it is out to loop back. I'm also hoping others will test/report/try the same tests with their V3. I burnt a few days trying to get to the bottom of this.... after spending $1500... so I'm honestly, not that keen to spend a lot more time on this one. I'll do the FW update and revisit this when I get a chance.
To be honest it's not a problem, especially now Bluetooth smart is available. I use powermatch and control the trainer with the pedals. Obviously just an erg mode issue.
Today I made a ride using the Tacx iPad app to control the trainer, performing a workout through manual edit of slope percentage. I had the Garmin Edge 1030 to measure power, cadence and speed through the Assioma's. The difference ei every clear, which you can see in the pictures attached through the links below.
From having discussed this now for a couple of days in different bike forum it is clear that I have to take the mental blow of accepting that my FTP is not as high as the Tacx values has shown earlier. It is about 50 W lower, judging by a new FTP test performed using the pedals for power metering. The support reply by Tacx is "update all firmware and clean it with white spirits and try again...".
https://teamscopum.se/wp-content/uploads/2018/02/Garmin_Assioma_-_6_10-5_5-5__60_100_rpm___Ride___Strava.png
https://teamscopum.se/wp-content/uploads/2018/02/Tacx_-_6_10-5_5-5__60_100_rpm___Ride___Strava
Original ride files is found here:
Garmin/Assioma:
https://www.strava.com/activities/1419964555/analysis/1539/2076
Tacx:
https://www.strava.com/activities/1419964376
The curves follow each other pretty well, but the difference in power levels are around 30 W in the chosen area in the pictures above.
Mental restart, and a bounce back to forget the old FTP value... :-)
It's the trainer, not the pedals.
I have the same issue with Tacx Neo and Vector 2 - 3.6% difference.
The Neo is always 3.6% lower than the Vectors.
What concerns me with the Neo is no provision for calibration, all components wear and that wear must affect results. Everything about the Neo is 'calculated' power, there is no flywheel of any meaningful mass, all power calculations are made from the amount of current used to create sufficient flux at the magnets and effect resistance. (The inverse is used for downhill simulation.)
BruceL have you considered drivetrain losses? Looks to me like they are reading extremely close and certainly within acceptable limits when you consider they are measuring power at different locations and as result there are power losses between the two points
I have an update for the vectors. First battery change and the left pedal died because the battery cover threaded. Looking at the forums this is a very common problem. Garmin have offered a replacement but probably going to have to wait 5 weeks. I can't believe that these pedals that cost £850 could be let down by such a cheap fitting. Garmin have a reputation for releasing items and using customers as Beta testers. I'm really frustrated by them.
Wonder what has occurred. I also did a concurrent test of Tacx Neo and Garmin Vector 3 pedals. The pedals always show 15 to 20 watts more power output than the tacx Neo. I am more inclined to trust the pedals. My impression of riding outdoors with significant hills is that power over 200 watts seems easier than indoors on the Neo. In other words, the Neo rides "heavy"--more perception of effort and lower wattage output reported.
Found this while looking into why the Tacx Neo we've borrowed is reading 30-40W lower than expected when compared to Kickr, Wattbike, SRM, Quark. Looks like the Neo is not as accurate as claimed, with many complaining (here, Tacx forums, Slowtwitch, Zwift) of it reading low by a wide margin but also just as many saying it's high. I've done a number of tests and all of them show the Neo reporting low and making that low power feel a lot harder to boot. I've even ridden Neo-Kickr-Wattbike one after the other and the Neo undoubtedly need more work to reach the same numbers. I've even ridden the Wattbike straight after session to exhaustion on the Tacx and found it easy to continue that power (and higher) on the Wattbike. Oh, and our Wattbike matches both what's expected for resistance/cadence combos, as well as matching 20 other Wattbikes I've tested.
I'm across all of the social groups.... except the ladies only one this topic has lit up over yesterday (yes, I've been queried a few times for reference over the post). I have not seen any conclusive evidence the Neo is reading high/low other than people comparing the data to other no-so-trustworthy sources (Gen I Stages, poorly torqued Vector1/2, etc).
The Neo can feel harder to people who've ridden other trainers due to the virtual flywheel. That's nothing to do with the power though.
Unless you've got the data, solid testing protocols, and have put them though a good analysis tool, then it's all opinion at this point. Which is what the Internet loves.... but engineers ignore. Case in point - "The Kickr is bad as power accuracy" - which is often spouted on forums... in fact, it's not. There were issues with Gen I Kickr unit accuracy, since resolved in firmware, and the Kickr is two hardware revisions down the track. However, people still love to recite that claim without ever testing it themselves or questioning the source of it.
It's interesting you've come across my post about the Neo readings being different to my Vector3 pedals (in one test case). I've done at least 20-30 posts/videos where the Neo has been SPOT ON correct with other power meters.
My conclusion based on 100s of hours across more than one Neo, and seeing their factory calibration process in person - is the Neo you've borrowed is the problem. Try another.
After buying a Tacx Neo that put me at 4 power measurement devices in my household. Since I got the Direto in February, it was a tad high but I didn't worry too much about it.
I use the CX bike on the Direto that has a Stages and they were close; within 5 watts or so. However, after getting my Neo this week it was noticeably low. So I ran some tests in a wattage comparison app and using 215 watts as a low base its close to this:
Neo = 215w (just purchased)
Vector 2s = 218w (used outdoors)
Stages = 230w (used on Direto)
Direto = 235w (used indoors)
After researching the best way to find a which one is accurate, I decided on the Vector Power Meter Advanced Torque Test below. My other option was to have someone with an SRM try out the Smart Trainers (which maybe I'll still do).
I followed the test protocols, and like any great mathematician, I showed my work:
Mass (kg) 29.4 lb = 13.33562 kg
Gravity = 9.81
Length (m) 170mm = .170m
Mass * Gravity * Length = 22.239813474 (expected result)
Left test = 22.22, 22.22, 22.25 (22.23 avg) Spot on!
Right test = 22.04, 21.91, 22.08 (22.01 avg) Close Enough?
After that I'm more or less happy with my Vector 2s; I believe that should put me within 1% on the right (I used an online calculator, got lazy)! The Neo is close enough I believe it is accurate as well, and that gap is just drivetrain losses.
What is annoying is 50% of my devices are off; the Direto will go to the wife and I may just sell the Stages.
Hi Shane - are you still using the Vector 3 pedals? Any updates since your last post?
Thanks
Soon.... waiting on Garmin for a solution to the battery housing issues.
Hi Shane,
I'm having a similar occurrence with Assiomas and a KICKR with them reading about 20W off from each other. Mine doesn't seem to be cadence related but I'm not entirely sure my testing protocol is accurate.
My first test was on Zwift doing a free ride. Pedals paired to Garmin 1000, KICKR to Zwift as power source and controllable trainer. I performed a zero offset on the Assiomas and then started riding. They were about 10W off. Then after 10 mins I did a spindown on the KICKR and that brought them closer. Discovered I needed to set the crank length in the Garmin (did in Assioma app but didn't realize that wouldn't transfer to head unit - RTFM!. After that their averages after 10mins was .5 watts difference. I can live with that, however looking at raw data they still had 5-10w difference in spots (looks almost like there is a little time lag in recording from one device).
Second test was using ERG (Jons Short Mix). Pedals paired again to Garmin & KICKR to Zwift. To start I was peddling between 140-160w and Zwift just kept showing me 160 on the nose. Then I would burst over so the pedals would spike and then come down and Zwift still didn't change from 160w. I guess that is really the KICKR in ERG keeping itself at that wattage no matter what I do with the pedals. So after about 20 mins of this game I put the KICKR on the Garmin and the Assioma as the power source in Zwift with the KICKR as the controllable trainer. This seemed to make things all right in the world, at least viewing between Garmin and Zwift real time (3 sec average on Garmin & Zwift were no more than a couple of watts at a time). However since I did this in the middle of a workout all the data is skewed after that point so it's hard to definitively say for sure right now. Will test this set up again tomorrow.
I'm thinking there are 2 demons here... ERG mode & Zwift recording of data. I'm guessing when you compare meters you are using different head units or applications and not data recording by Zwift? Other ideas? Do you have any testing protocol tips?
Jason
There's a lot going on here... start with ensuring ERG mode soothing is disabled on the Kickr.
Well that explains a lot! Unfortunately that setting isn't available for the KICKR 1st gen (according to a Sufferfest article), so I guess I'm SOL in that respect. If that is the case then I'll just do another free ride comparing the two to validate my data again doing a spindown and zero offset on each after 10 minutes.
Check the settings in the app just to be sure.
Thanks, did that first, then went searching why I didn't have the setting and found that SF article. Also checked to make sure there are no app updates and re-paired the KICKR with the app to be sure.
Happy Days! A factory spindown on the KICKR is what the doctor ordered. That brought everything into line with only a few watt difference (within their +/- tolerances) at most between the Assiomas and the KICKR. Avg power was different by 1 watt, and max by 5 watts, again both within +/-1%. Thank you for your help. Now to go check my ego and gain back my lost wattage thanks to my persnickety KICKR. ;)
Post a Comment