Findings report · 30 July 2026
We collected 380 ideas from seven forums, chose ten requests, and added one finding about the board itself. Then we built each one into a working mockup and filmed the result. The exercise showed which requests worked as written, which needed a different solution, and which carried consequences beyond the screen. It also gave us a demanding real-world test for Nolan.
Votes show where there is interest. They do not show what a change will touch.
The same pattern appeared across the ten requests we built. A vote count can show that people want a problem addressed, but it says little about the work or trade-offs involved. The board sorts requests by votes, so it is easy to read a high number as a reason to build something next. But the ranking did not predict which ideas were simple, which needed a different solution, or which affected other parts of the product.
The results do not follow the vote count. Requests with 694 and 142 votes both came with wider trade-offs. Requests with 444 and 103 votes both worked much as written. Adding a second approval level to bills was the largest change in this set, even though it had the fewest votes. It would affect bill statuses, audit history, and bills that had already been approved.
That makes sense once the two measures are separated. Votes reflect interest in a problem. The effort and risk depend on how much of the product a change touches. Looking at votes alone leaves that second part of the decision unexplored.
Two landed on “works as requested”, four on “needs a different solution”, and five on “wider trade-offs”.
Before making the mockups, we agreed that each walkthrough would end with one of three results. We could build it as requested, keep the problem but change the proposed solution, or show a consequence that would need a product decision.
Payer names on a reconciliation row and a default for the Approve button are both small changes with clear precedents elsewhere in the product. Neither required a larger redesign. The Approve default, for example, is a setting and a button-label change that makes the safer action easier to choose.
In these cases, the problem was clear but the suggested interface change would not do enough to solve it. Payment method on bills is the clearest example. Another column would display the method, but the mistake described in the request happens earlier. The mockup moves the decision to the point where the person can still prevent the error.
These requests are possible to build, but each one affects more than the original description suggests. Unapproving a sales invoice can leave Xero out of step with a document the customer already holds. Custom fields would be of limited use if they appeared in the editor but not on the PDF sent to the customer. Splitting a batch payment changes the identity of a payment after a remittance advice may already have been sent.
Two requests also overlap with work Xero has already committed to. Switching autosave off sits awkwardly beside a separate plan to move the Save button to the bottom of the invoice. That idea is In development with 247 votes. Xero has also Accepted an unapprove option for bills, so the question is why the same action may be suitable for bills but more complicated for invoices.
Our sample contained closely related requests with separate vote counts and different statuses.
We found this while comparing requests across the seven forums in our sample. The board ranks ideas by votes within each forum. When the same need is filed in two places, each version has its own page, vote count, and status. The two requests can even have different statuses without either page referring to the other.
Simply merging similar requests and adding their votes would create a new problem: it would treat different requests as if everyone had supported the same proposal. Software can help find possible matches, but it cannot make that judgement reliably. Our comparison of 380 titles found roughly fifty possible pairs. After reviewing them, we judged about six to be close matches. Small changes to the matching rules also changed which pairs appeared.
The useful improvement would be to show related requests together. That would let people see when the same need appears in several places, and when those requests have received different answers, without pretending they are identical.
The mistakes began when we started adding, ranking, and generalising.
Before publishing, we checked roughly 190 source details: vote counts, statuses, titles, links, forum names, and 36 direct quotations. None of those source details was wrong.
We were much less reliable when we started combining those facts. Of 38 statements involving totals, rankings, group sizes, or percentages, ten were wrong. Six of eight claims about the structure of the dataset were also wrong. We made four incorrect claims that something was the highest or largest, and another six went beyond what our sample could support. In three places, we called one request the highest-voted in the sample when it was actually second.
The mistakes appeared when we moved from facts to comparisons. A total, ranking, or claim about the largest item depends on what has been included. Our sample contains 380 of roughly 6,000 ideas, so it cannot support a claim about the whole board. We now state the limits of the sample alongside the numbers instead of leaving the reader to infer them.
Nolan turns a short screenplay into a clear, repeatable walkthrough of a real web product.
A feature can be working in code long before everyone understands what changed. Tickets, diffs, and release notes still ask the reader to imagine the interaction. Nolan gives an agent another option: show the change in the product and explain it in the same pass.
For this project, Nolan drove eleven working mockups in a real browser and turned each one into a captioned walkthrough. The screenplays determined what to show and say. A shared style kept the pacing, captions, colours, and presentation consistent across the set.
The result is not a set of one-off recordings. The screenplays and presentation remain editable, and Nolan can check whether a walkthrough's buttons and fields still exist before it is filmed again.
Our first screenplays put the background to each request into timed captions. That made the interface compete with the explanation for the viewer's attention. Moving the background to an opening title frame reduced the planned average from 87 to 65 seconds and left the captions free to describe the action on screen.
| Version | Average | Longest |
|---|---|---|
| Background in captions | 87s | 113s |
| Background on an opening frame | 65s | 75s |
| Final published films | 67s | 79s |
The timing result came from changing the direction, then generating the films again. The walkthroughs remain source-controlled work that we can review, test, and improve. See Nolan on GitHub or run the example.
Each page includes the original problem, the filmed walkthrough, and the reasoning behind our response.
| Request | Votes | Board status | What we concluded | Film |
|---|---|---|---|---|
| Payer names instead of “multiple items” | 103 | submitted | Works as requested | 61.4s |
| Select default for the Approve button | 444 | Not in pipeline | Works as requested | 58.3s |
| Payment method on bills | 222 | submitted | Different solution | 69.3s |
| Add subtotals | 348 | Not in pipeline | Different solution | 63.6s |
| Search across all bank accounts | 250 | Not in pipeline | Different solution | 61.7s |
| Demand split across forums | — | Not on the board | Different solution | 68.2s |
| Multiple levels of approval for bills | 142 | submitted | Wider trade-offs | 78.7s |
| Split a batch payment when reconciling | 345 | Not in pipeline | Wider trade-offs | 70.6s |
| Option to switch autosave off | 480 | Not in pipeline | Wider trade-offs | 68.9s |
| Unapprove option | 518 | Not in pipeline | Wider trade-offs | 71.6s |
| Custom fields on invoices and contacts | 694 | Not in pipeline | Wider trade-offs | 68.1s |
A vote is evidence of interest, not a product decision.
A vote shows that someone wants a problem addressed. It does not tell us how serious the problem is, whether the suggested solution is the best one, or what else would need to change. Building the requests gave us a way to investigate those questions.