Outsourcing QA Automation to India: What to Ask

Outsourcing test automation to India is common, and it can work very well. It can also leave you, six months later, with a folder of scripts nobody understands, a CI job that's always red, and a team that has moved on to another client. The difference is mostly decided before the contract is signed, by what you ask for.

We're an Indian team that builds and runs automation suites, so take this with that in mind. These are the questions we'd want a client to ask us.

What you're actually buying

You're not buying test cases or hours. You're buying a suite that runs on every change, tells you clearly when something is broken, and stays reliable as the product changes. Write that into the scope:

• which user journeys are covered, ranked by business risk
• where the tests run (your CI, on every pull request or nightly)
• what a failure report looks like and who gets it
• how long a full run takes
• who fixes tests when the product changes, and how fast

If a proposal counts test cases instead of journeys covered, ask why.

Ask for samples

Before choosing a team, ask to see:

1. A sample repository, even a cut-down one, so you can judge structure, naming and how test data is handled.
2. A CI run with its report, including what happened when a test failed.
3. Their flaky test policy. Every suite has flaky tests. Ask how they find them, whether they gate builds on them, and how long a test can stay in quarantine. Our post on fixing flaky Playwright tests shows what a good answer looks like.

Who owns what

Put these in the contract:

• the test code lives in your repository, under your organisation, from day one
• CI configuration and secrets are in your accounts, not the vendor's
• test data and any tools bought for the project are yours
• documentation good enough for another team to take over

This is the same principle as owning your store or software when an agency builds it. We wrote about it in what you should own.

Browsers and devices

Agree on coverage up front. Playwright, which we use for most web projects, supports Chromium, WebKit and Firefox on Windows, Linux and macOS, with mobile emulation for Chrome on Android and Mobile Safari. Emulation catches most layout and behaviour problems, but not all. Decide which journeys also need checking on real devices, and how often.

Data protection

Testers often need data that looks like production. Under India's DPDP Act, a vendor that processes personal data for you is a Data Processor, and Section 8(2) of the Act says you can only use one under a valid contract. Better still, don't give testers real customer data at all. Use generated test data, and mask anything copied from production.

Is automation worth it yet?

Automation pays off when the same journeys are tested again and again: frequent releases, a stable core flow, and a real cost when it breaks. If your product changes shape every week, or you release once a quarter, a smaller manual regression pass may be better value for now. Our post on when test automation pays off walks through the maths.

A good first month

A sound engagement starts small: the three to five journeys that cost the most when they break, running in your CI on every change, with a clear report. Once that's stable for a few weeks, expand. If a vendor wants to automate everything at once, they're selling hours.