Introduction

Monitor your NodeJS LLM Application with DokuMetry

DokuMetry Node SDK is the official library from Doku to send LLM usage data from your NodeJS application to Doku.

Installation

shell

npm install dokumetry

Usage

OpenAI

openai.js

import OpenAI from 'openai';
import DokuMetry from 'dokumetry';

const openai = new OpenAI({
  apiKey: 'My API Key', // defaults to process.env["OPENAI_API_KEY"]
});

// Pass the above `openai` object along with your DOKU URL and API key and this will make sure that all OpenAI calls are automatically tracked.
DokuMetry.init({llm: openai, dokuURL: "YOUR_DOKU_URL", apiKey: "YOUR_DOKU_TOKEN"})

async function main() {
  const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'What are the key to effective observability?' }],
    model: 'gpt-3.5-turbo',
  });
}

main();

The following OpenAI Endpoints are supported for monitoring with DokuMetry

• Chat completions
• Image Create and Image Variations
• Audio Speech Create
• Embeddings
• Fine Tunings
• Completions

Cohere

cohere.js

import { CohereClient } from "cohere-ai";
import DokuMetry from 'dokumetry';

const cohere = new CohereClient({
    apiKey: "YOUR_API_KEY",
});

// Pass the above `cohere` object along with your DOKU URL and API key and this will make sure that all Cohere calls are automatically tracked.
DokuMetry.init({llm: cohere, dokuURL: "YOUR_DOKU_URL", apiKey: "YOUR_DOKU_TOKEN"})

(async () => {
    const prediction = await cohere.generate({
        prompt: "hello",
        maxTokens: 10,
    });
    
    console.log("Received prediction", prediction);
})();

The following Cohere Endpoints are supported for monitoring with DokuMetry

• Chat
• Generate
• Summarize
• Embed

Anthropic

anthropic.js

import Anthropic from '@anthropic-ai/sdk';
import DokuMetry from 'dokumetry';

const anthropic = new Anthropic({
  apiKey: 'my api key', // defaults to process.env["ANTHROPIC_API_KEY"]
});

// Pass the above `anthropic` object along with your DOKU URL and API key and this will make sure that all Anthropic calls are automatically tracked.
DokuMetry.init({llm: anthropic, dokuURL: "YOUR_DOKU_URL", apiKey: "YOUR_DOKU_TOKEN"})

async function main() {
  const completion = await anthropic.completions.create({
    model: 'claude-2',
    max_tokens_to_sample: 300,
    prompt: `${Anthropic.HUMAN_PROMPT} how does a court case get to the Supreme Court?${Anthropic.AI_PROMPT}`,
  });
}

main();

The following Anthropic Endpoints are supported for monitoring with DokuMetry

• Completions

Parameters

Parameter	Description	Required
llm	Language Learning Model (LLM) Object to track	Yes
dokuURL	URL of your Doku Instance	Yes
apiKey	Your Doku API key	Yes
environment	Custom environment tag to include in your metrics	Optional
applicationName	Custom application name tag for your metrics	Optional
skipResp	Skip response from the Doku Ingester for faster execution	Optional

Advanced Usage

When using DokuMetry in Production environments, It is recommened to set skipResp: True for faster porcessing.

To filter your LLM Usage according application or the environment, set the application and environment parameters with the actual values.

Requirements

NodeJS >= 18.0 is supported.

If you are interested in other runtime environments, please open or upvote an issue on GitHub.