So i’m mostly a backend dev but i’ve been on this AR filters project for a couple months now, handling the android side of things since that’s apparently my job now. We’d already gone with a face tracking SDK early on, so I just started building on top of that.

Thing is, we know that part of our actual users are on old budget phones, not flagships. And that mattered a lot for this specific feature, because if the filters don’t run properly on those devices, it’s not like users get a slightly worse experience, they basically can’t use the feature at all. So testing on old android was a necessary part of the testing process.

Seemed fine initially but then we actually put it on the test devices and it was rough. The tracking does this whole 2D points thing and then tries to estimate 3D head pose from that, which sounds fine on paper but the second you moved your head fast or put your hand near your face it would just lose it. The filters would float off to the side. Really awkward looking.

Also their segmentation only did hair, which we didn’t realize until we were already pretty deep into testing. We needed skin retouching for like 40% of the features we’d planned so that was a problem.

So then we’re in this spot where we’re halfway through and someone’s like should we really switch SDKs now. It wasn’t a quick call, because ripping out something you’ve already half-integrated is never fun. But the segmentation issue alone meant we couldn’t hit what we’d promised, so we started looking at what else was out there specifically for our case, low-end device performance being the main thing we cared about. Instead of doing 2D then estimating 3D it just builds the 3D head model directly. Supposed to be lighter on resources and handle occlusions better, which on paper was exactly the thing we were missing.

When we tested it with the same put your hand on your face stuff it did hold up better. More stable. Switching mid-project wasn’t exactly smooth either, we’d already built up some muscle memory with the other SDK’s way of doing things, so there was a bit of a learning curve getting used to the new docs. But it fixed the segmentation issue, we could do skin stuff and not just hair, so we got those features back.

Feels like device support is always the thing that gets deprioritized until it’s not. Like everyone focuses on making the AR effect look impressive, but if it doesn’t hold up on the phones actual users have, it’s not adding anything for a chunk of them, it’s just making the app feel janky. How do you all catch this before it ships?