Every year we get onboarded to salvage a project that a different agency started and abandoned. The pattern is depressingly consistent: charismatic sales cycle, mediocre discovery, a lead engineer who leaves in month two, a scope that quietly balloons, and by month six the client has a half-built product and a very expensive lesson in agency selection.
There is no shortcut past this problem, but there is a checklist. Here is how a senior technical buyer should evaluate a custom software development company before signing anything.
1. Who will actually write the code?
The single most valuable question in the entire buying cycle. Ask by name. Ask for their LinkedIn. Ask how many years they've been at the agency. If the answer is 'we'll assign the team once we sign the SoW', walk away. Great agencies will name the engineer, the tech lead and the PM on the first sales call. Weak ones will not.
2. What percentage of their staff is senior?
This is the single strongest signal in the market. Agencies that scale by hiring cheap juniors and billing them out at senior rates are the ones producing the disasters. If less than 60% of engineers on client work have 6+ years of experience, expect a rough project. At Origami we are 100% senior on client work — no juniors, no exceptions. It is our single most important operating rule.
3. How often do they ship to production?
Weekly is the target. Every-two-weeks is acceptable. Monthly or less is a red flag: it means either the CI/CD is bad, the tests are missing, or the client is being kept in the dark about progress. Ask to see the last four release notes from a comparable client project (redacted is fine).
4. Can you talk to a live client?
Not a case study. A phone call with the person actually paying the invoices right now, ideally at a comparable-stage project. The best agencies are proud of their client relationships and will make the intro fast. The worst will 'circle back'.
5. What does their eval / observability practice look like?
For any AI-adjacent work, the eval story matters more than the model choice. Ask: 'show me the eval harness for a recent LLM feature you shipped'. If they don't have one, they are shipping demos, not products. The same question applies to non-AI: how do you monitor a live feature after launch? What's your rollback story?
6. How do they handle scope creep?
There is no such thing as a custom software project without scope creep. What matters is how the agency talks about it. Two red flags: (a) 'we do fixed-scope, fixed-price — everything else is a change request' (they will nickel-and-dime you); and (b) 'we don't track scope, we just build until you're happy' (they will churn on politics and take twice as long).
The healthy answer is: we run T&M or dedicated pods against a living backlog, we track velocity, we surface scope pressure weekly, and we make the trade-off explicit — cut something or add capacity. Grown-ups on both sides.
7. What's their staff turnover?
Ask directly: 'how long has the average engineer been at your company?'. Under 12 months means you'll get someone new mid-project. Over 3 years means the culture is retaining talent. At Origami our senior engineers average 3-5 year tenures — because we treat them like adults and pay them competitively.
The takeaway
A great custom software development company looks boring from the outside — same senior engineers, weekly releases, transparent tools, dull retrospectives. That boringness is the value. Everything exciting in this space is a warning sign.
If you want to see how Origami handles this — from senior-only staffing to weekly release cadence to eval-first AI — see how we approach engagements or book a scoping call.