Building SayThread, our dictation app for Windows and Linux

Our teams build software for clients. SayThread is one of our own products, alongside Neptune EHS. It turns speech into finished writing in 27 languages. It runs on Windows and Linux, and it is in beta.

This note covers what we built into it and what the build taught us about voice.

A transcript is not the goal

A plain dictation tool gives you a transcript, every word you said in the order you said it. That is rarely what you wanted to send. You still have to cut the filler, fix the order and set the tone. By the time you've done that, typing might have been faster.

So we aimed SayThread at the step after the transcript, which is writing you can send. On Windows and Linux it does four things:

• Dictate in any app
• Untangle what you said
• Rewrite by voice
• Get recaps of your meetings

Dictate in any app matters on a computer, where people write in email, chat, documents and forms. A dictation tool that only works in its own window adds a copy and a paste to every message. SayThread types into whatever field you were in, and reads the text before the cursor so it can carry on your sentence. It never reads passwords or terminals.

People don't speak in finished sentences

Say it messy. SayThread untangles it.

Untangle removes the ums, repeats and false starts. Spoken first drafts are full of them. If an app types them out word for word, the user spends the time they saved deleting them.

We treat cleanup as part of dictation itself, which is why Untangle is one of the four features.

People mix languages

SayThread works in 27 languages, and it keeps your English when you mix it with Urdu or Hindi. People who switch between English and Urdu or Hindi often do it inside a single sentence. A tool that expects one language at a time makes them pick one, or fix the result by hand. We built this support around how our own team in Rawalpindi speaks.

The lesson for any voice product is to test with the speech your users produce, mixed languages included. Clean sample sentences hide the hard cases.

Changing the tone without retyping

The same message can need a different tone for a client, a manager or a friend. Rewrite handles that without a retype.

On Windows and Linux, Rewrite by voice works on text you already have. You select the text in any app, hold SayThread's command key and say how you want it changed. For the common changes, SayThread's history has four styles: Fix grammar, Formal, Friendly and Gen-Z.

Gen-Z suits a quick message to a friend. The same words would read as careless in a note to a client, and Formal covers that case. The ready-made styles handle the frequent changes, and the voice command covers everything else.

Meeting recaps

SayThread records a call or a meeting, then turns it into text and a recap with a summary, a to-do list and chapters. Few people replay a whole recording to find one decision. The summary says what happened, the to-do list says what comes next, and the chapters take you to the part you need. The full text is still there when you want to check the exact words.

Building for Windows and Linux

SayThread runs on Windows and Linux, and both apps are in beta.

Both apps come from one .NET codebase with a shared Avalonia UI. The Windows app adds native pieces for the keyboard hook, audio and inserting text.

A desktop dictation app sits next to everything else a person has open. It has to start listening the moment you hold the key, and put the text where you were already typing. If you'd rather not hold a key, you can switch to hands-free.

Getting text into other apps works differently on each system. SayThread types into the focused field by simulated typing or by a paste, whichever that app accepts. On Windows it reads the text before the cursor through UI Automation, and on Linux through AT-SPI, the desktop's accessibility service; both are what screen readers use, so a dictation picks up the names already in the sentence. On Linux it types through a virtual keyboard.

What the build taught us

• A transcript is halfway. People judge a voice app by the text they end up sending, so the work after transcription is the product.
• Real speech is the test case. Filler words, false starts and two languages in one sentence are normal input. Build for them first.
• Common choices should be ready-made. Rewrite's styles cover the frequent changes, and the voice command covers the rest.
• Being our own client helps the client work. On SayThread we make the scope calls and live with them, which is the position our clients are in on every build.

If you're adding voice to your own product

These are the questions we'd ask about any product that takes speech as input:

1. Where does the text end up? If the answer is another app, the feature has to reach that app, or the user copies and pastes every time.
2. What does the user do with the text next? If they always clean it up, the product should do the cleanup.
3. Which languages do your users speak, and do they mix them? Test with recordings of real users, not only with read-aloud scripts.
4. How does the user fix a mistake? A voice feature that gets a word wrong needs a quick way to correct it, by voice or by hand.

None of these questions is about the speech engine. They are about what the person is trying to send.

Where it stands

SayThread is in beta, and saythread.com has the latest on it. If you're adding voice to a product of your own, we're glad to compare notes.