SE Radio 703: Sahaj Garg on Low Latency AI

In this episode, Sahaj Garg, CTO of wispr.ai, joins SE Radio host Robert Blumen to talk about the challenges of building low-latency AI applications. They discuss latency's effect on consumer behavior as well as interactive applications. The conversation explores how to measure latency and how scale impacts it. Then Sahaj and Robert shift to themes around AI, including whether "AI" means LLMs or something broader, as they look at latency requirements and challenges around subtypes of AI applications. The final part of the episode explores techniques for managing latency in AI: speed vs accuracy trade-offs; speed vs cost; latency vs cost; choosing the right model; reducing quantization; distillation; and guessing + validating.

Brought to you by IEEE Computer Society and IEEE Software magazine.

In this episode, Derick Schaefer, author of CLI: A Practical Guide to Creating Modern Command-Line Interfaces, talks with host Robert Blumen about command-line interfaces old and new. Starting with a short review of the origin of commands in the early unix systems, they trace the evolution of commands into modern CLIs. Following the historic rise, fall, and re-emergence of CLIs, they consider innovative examples such as git, github, WordPress, and warp. Schaefer clarifies whether commands are the same as CLIs and then discusses a range of topics, including implementation languages, packages in the golang ecosystem for CLI development, CLIs and APIs, CLIs and AIs, AI tooling versus MCP, the object-command pattern, command flags, API authentication, whether CLIs should be stateless, and output formats – json, rich text.

Brought to you by IEEE Computer Society and IEEE Software magazine.

Show Notes

Related Links

• Book: >CLI

• go pFlag package: GitHub – spf13/pflag: Drop-in replacement for Go’s flag package, implementing POSIX/GNU-style –flags.

• go Cobra package GitHub – spf13/cobra: A Commander for modern Go CLI interactions

• Derick Schaefer on LinkedIn https://www.linkedin.com/in/derickschaefer/

• se-radio episode SE Radio 679: Wesley Beary on API Design

Transcript

Original source