Announcing Together Python SDK v2.0

Today we’re shipping the Python SDK v2.0 Release Candidate, a new, type‑safe, OpenAPI‑driven client for Together’s API — designed to be faster, easier to maintain, and ready for everything we’re building next.

The beta is an opportunity for developers to test out the SDK and ensure the new release does not introduce any regressions for their particular systems and app configurations. We will be continuously releasing fixes and improvements during the beta period — some of these may include breaking changes.

Why a new Python SDK?

The core reasons:

• Modern architecture
  The new SDK is generated from our OpenAPI specification using Stainless, giving you a closer 1:1 mapping to our API, fewer edge‑case inconsistencies, and a much easier path for us to ship new features quickly.
• Better type safety & DX
  The SDK brings “TypeScript‑like” typing to Python, including typed parameters, responses, and helpers for chat messages and eval parameters. This makes it easier to build large codebases with confidence and better editor support.
• Future‑proof & feature‑first
  Active development and new API features will land in v2 first. After the RC period is over, v1 will move into maintenance and then deprecation
• `uv` Support
  Compatible with uv, the fast Python package installer - `uv add together --prerelease allow`
• Modern HTTP client & performance.
  Under the hood, the new SDK uses httpx instead of requests, brings better timeout and connection handling, and in our benchmarks is about 20ms faster per request on average.

Getting started

1. Install the RC

    
    # Install uv
    curl -LsSf https://astral.sh/uv/install.sh | sh

    # Create a new project and enter it
    uv init myproject
    cd myproject

    # Install the Together Python SDK (allowing prereleases)
    uv add together --prerelease allow

    # pip still works aswell
    pip install --pre together
    

2. Skim the Migration Guide

Start with the Python SDK Migration Guide for an overview, API‑by‑API notes and code snippets, breaking changes, new error handling, and more.

The guide also includes a Feature Parity Matrix that breaks APIs into:

• ✅ Easy migrations: Chat, completions, embeddings, images, models, audio transcription/translation, many fine‑tuning flows.
• ⚠️ Medium effort: Files, fine‑tuning checkpoints/download, batches, endpoints, evals, code interpreter, some audio speech changes.
• 🆕 New capabilities: Jobs API, Hardware API, enhanced evals, code interpreter sessions, raw response helpers.

3. Try your existing workloads

• Begin with chat/completions/embeddings to confirm “no change” behavior.
• Then test Files, Batches, Endpoints, Evals, and Code Interpreter if you use them.
  

4. Report issues & suggestions

• ‍Open an issue in the repo with a minimal repro.
• Report issues on Discord.
• Tell us which APIs you’re using and whether you’re on RC in the report.

Key breaking changes to know about

The details live in the Breaking Changes section of the migration guide, but here are the highlights to keep in mind while you test the RC.

• Constructor parameters
  
  • supplied_headers → default_headers
  • New optional parameters: default_query, http_client, _strict_response_validation
    
• Error handling
  
  • TogetherException → TogetherError
  • Timeout → APITimeoutError
  • Several old exceptions removed or folded into APIStatusError / specific HTTP status classes
    
• Arguments
  
  • No positional parameters — all calls must use keyword arguments
  • **kwargs replaced by explicit extra_headers, extra_query, extra_body, timeout
    
• Types
  
  • Many response type names in together.types.* have changed

Files

• client.files.retrieve_content(...) is gone
• Use client.files.content(file_id) instead; it returns a streaming binary response, and no longer writes to disk by default

Batches

• Method renames:
  
  • create_batch → create
  • get_batch → retrieve
  • list_batches → list
  • cancel_batch → cancel
• Parameter change: file_id → input_file_id
• Response change: create now returns the full API response; you access .job yourself

Endpoints

• client.endpoints.get(...) → client.endpoints.retrieve(...)
• min_replicas / max_replicas are now nested under an autoscaling parameter in create
• List methods (list, list_avzones) now return an object wrapper instead of a bare array

Evals

• Namespace change: client.evaluation → client.evals
• evals.create now takes a strongly‑typed parameters object (e.g., ParametersEvaluationClassifyParameters) instead of a loose dictionary

Audio Speech

• Added client.audio.speech.with_streaming_response.create(...) method to stream audio out

Code Interpreter

• client.code_interpreter.run(...) → client.code_interpreter.execute(...)
• Result structure and sessions are updated (result.data.outputs[0].data, client.code_interpreter.sessions.list())

Hardware & CLI

• client.endpoints.list_hardware() is removed; use client.hardware.list() instead
• Some legacy CLI commands (like chat.completions, completions, images generate) are removed or re‑shaped in v2

What’s in the 2.0 RC?

The RC is feature‑complete for 2.0. We’re not planning major new features before GA; the goal now is polish, migration, and feedback.

APIs that “just work” (no code changes)

For most common workloads, you can upgrade with little or no code change. The following APIs are effectively drop‑in compatible between v1 and v2:

• `client.chat.completions.create`
• `client.completions.create`
• `client.embeddings.create`
• `client.images.generate`
• `client.rerank.create`
• `client.fine_tuning.create / list / retrieve / cancel`
• `client.models.list`

If your app is mostly centered around calling a model, streaming results, generating images, transcribing, and maybe fine‑tuning, the migration should be straightforward. You can look at the new error model and breaking changes section.

APIs with some changes

Other APIs are available in the RC, but with updated signatures or response shapes:

• Files (client.files.*)
• Batches (client.batches.*)
• Speech (client.audio.speech.*)
• Transcriptions/Translations (client.audio.translations.*)
• Endpoints (client.endpoints.*)
• Evals (client.evals.*, previously client.evaluation)
• Code Interpreter (client.code_interpreter.*)
• Some Fine‑tuning & Models helpers

These are all covered in detail in this migration guide section.

New SDK‑only APIs

The RC also introduces net new features that never existed in the legacy SDK:

• Jobs API (client.jobs.*) — general job management
• Hardware API (client.hardware.*) — discover and price hardware options
• Raw response & streaming helpers — for debugging, observability, and fine‑grained control

Net‑new features in Python SDK v2

1. New error model

The error hierarchy has been completely redesigned to be more precise and predictable.

Legacy (v1):

• Base: TogetherException
• Mixed bag of exceptions like ResponseError, JSONError, InstanceError, Timeout, InvalidRequestError, etc.

New (v2):

• Base: TogetherError (replaces TogetherException)
• HTTP errors: APIStatusError plus specific status‑code classes:
  
  • BadRequestError (400)
  • AuthenticationError (401)
  • PermissionDeniedError (403)
  • NotFoundError (404)
  • ConflictError (409)
  • UnprocessableEntityError (422)
  • RateLimitError (429)
  • InternalServerError (5xx)
    
• Transport & timeout:
  
  • APIConnectionError
  • APITimeoutError (replaces Timeout)
    
• Validation:
  
  • APIResponseValidationError
    
• Domain‑specific:
  
  • FileTypeError, DownloadError (new module paths).

2. Modern parameter & type system

The new SDK leans heavily into keyword‑only arguments and explicit optional semantics:

• All API methods use keyword‑only arguments, positional args are no longer supported.
• Optional parameters use a NOT_GIVEN sentinel (and the “omit” behavior it represents) instead of overloading None.
• “Extra” parameters are passed like this:
  
  • extra_headers
  • extra_query
  • extra_body
  • timeout

You also get much richer typing for messages and responses via together.types.*.

3. Jobs API

The Jobs API provides general job management capabilities that were previously only accessible via raw HTTP or cURL.

    
    job = client.jobs.retrieve(job_id="job-abc123")
    print(job.status)

    jobs = client.jobs.list()
    for j in jobs.data:
        print(j.id, j.status)
    

4. Hardware API

The Hardware API lets you discover and price hardware options for a given model:

    
    hardware_list = client.hardware.list(model="meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo")

    for hw in hardware_list.data:
        price_per_hour = hw.pricing.cents_per_minute * 60 / 100
        print(f"{hw.id} — ${price_per_hour:.2f}/hour")
    

This replaces legacy helpers like client.endpoints.list_hardware().

5. Raw response & streaming helpers

You can now easily access the raw HTTP response or use streaming with a context manager:

Raw response:

    
    response = client.chat.completions.with_raw_response.create(
        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
        messages=[{"role": "user", "content": "Hello"}],
    )

    print(response.status_code)
    print(response.headers)
    completion = response.parse()
    

Streaming with context manager:

    
    with client.chat.completions.with_streaming_response.create(
        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
        messages=[{"role": "user", "content": "Write a story"}],
        stream=True,
    ) as stream:
        for line in stream.iter_lines():
            print(line)
    

6. Code Interpreter sessions

The new SDK exposes session management for the Code Interpreter:

    
    result = client.code_interpreter.execute(
        code="print('Hello, world!')",
        language="python",
    )

    print("Output:", result.data.outputs[0].data)

    sessions = client.code_interpreter.sessions.list()
    for session in sessions.data.sessions:
        print("Session:", session.id)
    

RC period, expectations, and what we’re looking for

This Release Candidate period is focused on validation, not big new features. We’re shipping a feature‑complete RC and aim to gather real‑world feedback, adjust, then cut GA.

During the RC:

• ✅ We do expect:
  
  • Bug fixes
  • Documentation improvements
  • Small ergonomics tweaks
  • Limited, well‑documented breaking changes if needed
    
• 🚫 We don’t expect:
  
  • Entirely new API families
  • Large behavioral shifts in core APIs (chat, completions, embeddings, etc.)

We’d especially love feedback on:

• Migration pain points (anything that felt confusing or surprising)
• Error handling and type ergonomics
• Jobs & Hardware APIs
• Raw/streaming response utilities
• Performance characteristics in your environment

After the RC window, we’ll:

• Incorporate your feedback
• Lock down any remaining breaking changes
• Publish Python SDK v2.0 GA
• Announce a clear long‑term support and deprecation plan for v1

Your feedback via issues on the repo or discussion on Discord over the next month will directly shape the final 2.0 release.