Recently, I stumbled upon a tweet from the creator of Instructor, a Python library that has a great community. They were on the hunt for someone to craft the JavaScript version. The mission and vision align with what I have been working towards on my own, so I reached out and started building. I built most of Instructor on top of some of the tools I have been working on this year - enabling parital json streaming and managing structured output with Zod. Instructor has a nice clean API and an existing community that I am excited to start working with.

Instructor is similar to to what I did with "schema agents" in my agents package - but focused on structured extraction. "Structured extraction in Typescript, powered by llms, designed for simplicity, transparency, and control." The Instructor instance is a proxy directly to the OpenAI SDK - it only patches the chat completion call wtih a few new options - adding a response_model (a zod schema) and on the instance intiialization a "mode" - which determines wether or not to coerce the response to json via a prompt, function call, or function call via tools. The simplicity and focus on staying close to the sdk makes it approchable and clear.

The project fits well within the other work I have been doing so I am excited to keep contributing and work it into my stack.

I was able to use a lot of pre-exisiting tools I wrote from the past work I have done in the base Instructor instance and was able to enable a powerful streaming mode using schema-stream

The github here: instructor-js

Basic streaming example with instructor-js

---

Define a zod schema

export const ExtractionValuesSchema = z.object({
  users: z
    .array(
      z.object({
        name: z.string(),
        handle: z.string(),
        twitter: z.string()
      })
    )
    .min(5),
  date: z.string(),
  location: z.string(),
  budget: z.number(),
  deadline: z.string().min(1)
})

export type Extraction = Partial<z.infer<typeof ExtractionValuesSchema>>

make a completion call

import Instructor from "instructor"
import OpenAI from "openai"
import { z } from "zod"

import { extractionValuesSchema, Extraction } from "./schema"

const textBlock = `
In our recent online meeting, participants from various backgrounds joined to discuss the upcoming tech conference. The names and contact details of the participants were as follows:

- Name: John Doe, Email: johndoe@email.com, Twitter: @TechGuru44
- Name: Jane Smith, Email: janesmith@email.com, Twitter: @DigitalDiva88
- Name: Alex Johnson, Email: alexj@email.com, Twitter: @CodeMaster2023
- Name: Emily Clark, Email: emilyc@email.com, Twitter: @InnovateQueen
- Name: Ron Stewart, Email: ronstewart@email.com, Twitter: @RoboticsRon5
- Name: Sarah Lee, Email: sarahlee@email.com, Twitter: @AI_Aficionado
- Name: Mike Brown, Email: mikeb@email.com, Twitter: @FutureTechLeader
- Name: Lisa Green, Email: lisag@email.com, Twitter: @CyberSavvy101
- Name: David Wilson, Email: davidw@email.com, Twitter: @GadgetGeek77
- Name: Daniel Kim, Email: danielk@email.com, Twitter: @DataDrivenDude

During the meeting, we agreed on several key points. The conference will be held on March 15th, 2024, at the Grand Tech Arena located at 4521 Innovation Drive. Dr. Emily Johnson, a renowned AI researcher, will be our keynote speaker.

The budget for the event is set at $50,000, covering venue costs, speaker fees, and promotional activities. Each participant is expected to contribute an article to the conference blog by February 20th.

A follow-up meeting is scheduled for January 25th at 3 PM GMT to finalize the agenda and confirm the list of speakers.
`



const oai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY ?? undefined,
  organization: process.env.OPENAI_ORG_ID ?? undefined
})

const client = Instructor({
  client: oai,
  mode: "TOOLS"
})

const extractionStream = await client.chat.completions.create({
  messages: [{ role: "user", content: textBlock }],
  model: "gpt-4",
  response_model: ExtractionValuesSchema,
  max_retries: 3,
  stream: true
})

let extraction: Extraction = {}

for await (const result of extractionStream) {
  try {
    extraction = result
    console.clear()
    console.table(extraction)
  } catch (e) {
    console.log(e)
    break
  }
}

console.clear()
console.log("completed extraction:")
console.table(extraction)

return a completion stream from an api route

import { ReadableStream } from 'stream';
import { extractionValuesSchema, Extraction } from "./schema"

function asyncGeneratorToReadableStream(generator) {
  const encoder = new TextEncoder();

  return new ReadableStream({
    async start(controller) {
      for await (const parsedData of generator) {
        controller.enqueue(encoder.encode(JSON.stringify(parsedData)));
      }
      controller.close();
    },
    cancel() {
      if (cancelGenerator) {
        cancelGenerator(); 
      }
    }
  });
}


export const runtime = "edge"

export async function POST(request: Request): Promise<Response> {
  const { messages, prompt } = await request.json()

const oai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY ?? undefined,
  organization: process.env.OPENAI_ORG_ID ?? undefined
})

const client = Instructor({
  client: oai,
  mode: "TOOLS"
})

const extractionStream = await client.chat.completions.create({
  messages: [...messages, { role: "user", content: prompt }],
  model: "gpt-4",
  response_model: ExtractionValuesSchema,
  max_retries: 3,
  stream: true
})

  
  const stream = asyncGeneratorToReadableStream(extractionStream);


  return new Response(stream)
}