Skip to main content

Together AI

Here's an example of calling a Together AI model as an LLM:

import { TogetherAI } from "@langchain/community/llms/togetherai";
import { PromptTemplate } from "@langchain/core/prompts";

const model = new TogetherAI({
model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
});
const prompt = PromptTemplate.fromTemplate(`System: You are a helpful assistant.
User: {input}.
Assistant:`);
const chain = prompt.pipe(model);
const response = await chain.invoke({
input: `Tell me a joke about bears`,
});
console.log("response", response);
/**
response Sure, here's a bear joke for you: Why do bears hate shoes so much? Because they like to run around in their bear feet!
*/

API Reference:

info

You can see a LangSmith trace of this example here

You can run other models through Together by changing the modelName parameter.

You can find a full list of models on Together's website.

Streaming

Together AI also supports streaming, this example demonstrates how to use this feature.

import { TogetherAI } from "@langchain/community/llms/togetherai";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const model = new TogetherAI({
model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
streaming: true,
});
const prompt = ChatPromptTemplate.fromMessages([
["ai", "You are a helpful assistant."],
[
"human",
`Tell me a joke about bears.
Assistant:`,
],
]);
const chain = prompt.pipe(model);
const result = await chain.stream({});
let fullText = "";
for await (const item of result) {
console.log("stream item:", item);
fullText += item;
}
console.log(fullText);
/**
stream item: Sure
stream item: ,
stream item: here
stream item: '
stream item: s
stream item: a
stream item: light
stream item: -
stream item: heart
stream item: ed
stream item: bear
stream item: joke
stream item: for
stream item: you
stream item: :
stream item:

stream item:

stream item: Why
stream item: do
stream item: bears
stream item: hate
stream item: shoes
stream item: so
stream item: much
stream item: ?
stream item:

stream item:

stream item: Because
stream item: they
stream item: like
stream item: to
stream item: run
stream item: around
stream item: in
stream item: their
stream item: bear
stream item: feet
stream item: !
stream item: </s>
Sure, here's a light-hearted bear joke for you:

Why do bears hate shoes so much?

Because they like to run around in their bear feet!</s>
*/

API Reference:

info

You can see a LangSmith trace of this example here


Was this page helpful?


You can leave detailed feedback on GitHub.