The Blind Spot in My First Test Coverage Plan

A few weeks ago, I sat down to plan an automated test suite for a mobile app that, frankly, had none worth trusting. The previous attempt had been heavyweight, brittle, and eventually removed entirely, so I was essentially working from a clean slate.

My instinct was to start deep. I picked the areas that seemed to matter most: sign-in, the main list view, and the detail screen users spend a lot of time in. My plan was to build meaningful coverage around those flows first, including happy paths, error states, and edge cases, and then gradually expand outward once the critical parts were in good shape.

It felt like a reasonable approach. On paper, it looked fairly rigorous.

The Plan That Looked Solid

A long list of edge cases and negative paths around the highest-priority screens gives you the feeling that you're building something substantial. There was plenty of work in the plan, the scenarios were meaningful, and the areas I had chosen were absolutely worth protecting.

Then, while reviewing the plan with a principal engineer on the team, I got a very simple question:

If the dashboard breaks tomorrow, would any of this catch it?

It wouldn't.

The app had more than a dozen distinct functional areas, and I had chosen three because they were the most critical ones, giving them meaningful coverage while leaving the rest completely unmonitored. The problem wasn't that the selection of critical areas was wrong; those flows still deserved deeper coverage. The problem was that I had implicitly treated "most critical" as synonymous with "where I should start," rather than as just one dimension among others when deciding how to structure the initial coverage.

Once I saw that distinction, I changed the plan.

Depth Without Breadth Creates a Blind Spot

Going deep gives you confidence in the places you've already chosen to inspect. What it doesn't tell you is what's happening everywhere else.

A test suite with strong coverage around three screens and no coverage around ten others can still be valuable, but for those ten areas it is functionally no different from having no automated tests at all. A regression can make it into one of those areas and the suite has no opinion about it because I never gave it one.

That was the blind spot in my original approach. I had used importance to decide both where to start and where to invest the most effort, when those are actually two different decisions.

Importance is a good way to decide where to go deep. Lack of visibility is a better way to decide where to go first.

That distinction changed how I structured the entire project.

How I Changed the Plan

Instead of starting with deep coverage around a handful of flows, I reordered the first milestone around breadth. I mapped the major functional areas of the application and changed the goal so that each one would get a basic smoke test before I started adding layers of coverage to individual areas.

These tests weren't meant to be exhaustive. The goal was much simpler: does the screen load, can the primary interaction still be completed, and would the test suite tell us if this part of the application suddenly stopped working?

The deeper work didn't disappear. Sign-in, the main list, detail flows, error handling, and edge cases were still important and stayed in the project. I simply moved that work behind the initial breadth milestone.

I also moved CI earlier in the plan. Rather than waiting for the deeper suites to be complete before making automation part of the development workflow, I wanted the smoke suite running as soon as it started providing useful coverage.

The principle I ended up using was simple: give every major area one trip wire before giving any particular area its second or third one.

A dozen shallow tests can often be built faster than a few exhaustive suites, and once they exist, every change to the application has at least some chance of exposing a regression outside the areas I've already decided are important. It isn't comprehensive coverage, but it closes a much larger visibility gap surprisingly quickly.

The Question I Should Have Asked Earlier

The mental model I took away from this is simple: if this breaks, would we know?

That question changes the way I think about coverage because it moves the conversation away from test counts or how thoroughly one feature is exercised. Instead, it exposes parts of the system where failure can currently happen silently.

It also made me reconsider what I should have done before writing the coverage plan in the first place. Coming into an existing codebase, there is usually more context around testing than what is visible in the current test target. There may be abandoned automation, old test cases, regression checklists, historically fragile areas, or manual release processes that already tell you something about where failures tend to happen.

In this case, I moved into solution mode before fully reconstructing that picture. I had identified several important areas correctly, but I hadn't mapped the entire surface of the application well enough to understand what my proposal was leaving out.

Now I'd start with a slightly different set of questions. What does the team already know? Which areas currently have no automated signal at all? What could break without anybody finding out until someone manually happens across it? Once I have that map, then I can decide where deeper coverage will return the most value.

What This Changed for Me

The part I found most useful about this experience wasn't really about automated testing.

I had just joined the team, I was responsible for putting the testing strategy and project together, and I wanted to start contributing quickly. I saw a problem, formed a reasonable approach, and turned it into a concrete plan. That's generally a useful instinct, especially when you're expected to provide technical direction, but in this case it also meant I started shaping the solution before I had completely mapped the problem.

Experience helps you recognize patterns faster, but there is a trade-off there. The more patterns you've seen, the easier it becomes to look at a new problem and quickly decide what kind of problem it is. Most of the time that's useful. Occasionally it means you commit to a familiar shape before you've asked enough basic questions.

One question from another engineer was enough to expose the gap. From there, it was still my responsibility to decide what to do with that feedback. I went back through the scope, changed the sequencing, expanded the first milestone across the application, and adjusted the CI plan around it.

That's one of the things I value about working with strong engineers. Being responsible for the technical direction doesn't mean every useful idea has to originate with you. It means being able to recognize when someone has exposed a flaw in your reasoning and then turn that observation into a better plan.

The original plan wasn't bad. It was optimized in the wrong order.

Test what you know is important, eventually. Test everything at least a little, first. And before either of those, understand what's already known.