A team at Microsoft, including TypeScript inventor Anders Heljsberg, has introduced TypeChat, the aim being to add structure to conversational AI.

“The current wave of LLMs default to conversational natural language — languages that humans communicate in like English. Parsing natural language is an extremely difficult task,” states the post introducing the new library.

TypeChat is based on TypeScript types. The developer defines types that represent the kinds of question, or “intents”, that the application deals with. The TypeChat library will construct prompts to the LLM (Large Language Model) AI, such as OpenAI’s ChatGPT, that asks for data to be returned in a manner that conforms to the types. If the responses fail validation, it will attempt to repair it with further interaction. Finally, TypeChat will summarize the interaction so it can be confirmed before any action is taken. The data is transmitted in JSON format, since it turns out, the docs state, that “many language models are great at generating JSON.”

Examples provided by the team cover sentiment analysis for user input, food ordering for a café or restaurant, calendar scheduling, math calculations, and playing music on Spotify.

The benefits claimed are greater accuracy, more approachable programming, and greater safety since the types constrain the AI responses.

The introduction of TypeChat comes just a few weeks after OpenAI introduced “new function calling capability in the Chat Completions API.” This allows developers to “describe functions to the model via JSON Schema,” and have the LLM output JSON with arguments to call those functions. The idea is not dissimilar, since it means that the output from the LLM integrates with the developer’s code.

Does this make TypeChat redundant? The question has been asked on TypeChat’s GitHub repository; but part of the answer no doubt is that TypeChat is designed to work with any LLM, not just OpenAI – though currently all the examples offered by the team run on OpenAI or Azure OpenAI endpoints, which is not surprising given Microsoft’s close association with OpenAI.

Developer reaction is mixed, ranging from “can’t wait to try it” to “Why are we trying to get structured output out of something that was specifically designed to produce natural-language output?” There are also plenty of other projects which address the same problem, not least Microsoft’s own Guidance project.

The attraction of TypeChat though is that TypeScript is already familiar to millions of developers, and the seniority of the team behind it which includes Hejlsberg and also Daniel Rosenwasser, TypeScript senior program manager, Technical Fellow Steve Lucco, and others.

The problem perhaps is that the strong Microsoft and OpenAI flavour to the project so far may deter the kind of vendor neutral adoption that the the team says it would like to see.