Why Traditional API Testing Falls Short: A Comparison of Shadow, Production, and Replay Techniques
I want to share the story of how our team at a fast-paced startup tackled the challenge of API testing without any dedicated QA team, the roadblocks that we encountered, and how we ultimately addressed these issues. Baseline Challenge We had 15-day sprints including mandatory unit and API testing. Strict Timelines (2 weeks) 80% unit test and API test mandate Initially, we relied on automated testing. Unit tests were stable, but API scripts quickly outdated due to frequent data changes and complex scenarios. Lacking a clear evaluation metric like code-coverage in unit tests, we wanted to cover as many user-scenarios possible for APIs. I suggested the record-replay technique for creating disposable API tests. In short, for API tests I wanted: Close to real-world API tests (ideally from prod) Easy disposable tests - quickly create/discard Automatic Infrastructure spin-up or service/dependency virtualization.. Writing test automation script was not an option since we got only 15 days...