How to Run Muse Spark Open Locally

TL;DR

Running Muse Spark Open locally is possible, but it is a large-scale deployment rather than a typical single-GPU setup. Meta’s documented configurations include four NVIDIA B300 GPUs for NVFP4, eight B300 GPUs for BF16, or a distributed MLX setup using multiple M5 Ultra Mac Studios.

For most developers, the first step is to match the checkpoint to a supported hardware and inference-runtime combination. Start with a conservative context length, verify that the server returns complete responses, and measure memory usage before increasing context or concurrency. If you do not have access to suitable hardware, a hosted inference API may be the more practical option such as CometAPI's Muse Spark .

Key Takeaways

• Muse Spark Open has 816B total parameters and about 42B active per token; the full expert bank still needs storage and memory.
• Choose NVFP4 or BF16 using checkpoint size, accelerator memory, interconnect, and serving overhead.
• The NVIDIA example uses the current four-B300 NVFP4 vLLM configuration; verify your exact hardware and runtime before launching.
• The Apple route requires multiple M5 Ultra Mac Studios and RDMA over direct Thunderbolt 5 links.
• Verify the model endpoint, response completion, and memory usage before increasing context or adding users.

What Is Muse Spark Open?

Muse Spark Open is the downloadable-weights release of Meta’s Muse Spark family. Unlike a conventional dense transformer, its sparse MoE design routes each token through a subset of experts, supporting frontier-scale capacity with a smaller active computation footprint. Meta’s release announcement describes it as natively multimodal and optimized for multi-step coding and agentic workflows.

The model accepts text, images, audio, and video and produces text; feature availability still depends on the serving runtime. It is distributed under the Muse Model License with usage restrictions. Check the repository terms before deployment and review irreversible actions in agent workflows.

How Does Muse Spark Open Perform?

Quality benchmarks help establish whether the model is worth evaluating; they do not tell you how fast your local server will run. The figures below are Meta-reported capability results, not measurements from the deployment commands in this guide.

Meta documents model settings, benchmark-specific procedures, and score selection in its evaluation methodology. Muse Spark Open uses xhigh reasoning in these results. Compare a local setup using your own task set, software version, context, and concurrency; a lower served context or a different harness changes what is being tested.

Can You Actually Run Muse Spark Open Locally?

Yes. You can run Muse Spark Open locally if your system meets the checkpoint, accelerator, and inference-runtime requirements. The documented routes include NVIDIA GPU servers and distributed Apple Silicon systems; a compatible GGUF conversion provides another option for machines with enough addressable memory. Meta’s official Muse Spark Open model card and the Hugging Face deployment guide describe the released weights and runtime-specific setup. A standard consumer laptop cannot keep the full native checkpoint in memory.

Memory capacity is the first constraint. Active parameters describe per-token computation, not the complete expert-weight footprint. Meta’s checkpoint guide gives approximately 537 GB for NVFP4 and 1.65 TB for BF16, excluding KV cache, activations, and runtime overhead.

Allocate hardware using the full checkpoint size plus serving headroom. Long prompts add KV-cache pressure, and supported GPU counts depend on the runtime and accelerator. The four-GPU NVFP4 configuration is not a promise that any four Blackwell cards can serve every context length.

A private GPU server is the native-weight route. Multiple high-memory Mac Studios are a separate supported route. Consumer desktops require a compatible quantized build and careful capacity checks; a large SSD alone cannot substitute for fast inference memory.

Which Checkpoint Should You Choose?

Choose a checkpoint before downloading. Meta publishes BF16 and NVFP4 native checkpoints, but these are not universal format recommendations for every platform. The documented NVIDIA Blackwell routes use BF16 or NVFP4; AMD Instinct routes use BF16 or AMD-specific MXFP4/FP8 checkpoints; Intel Xeon 6 uses an INT4 configuration; and the distributed Apple Silicon route uses the NVFP4 checkpoint with MLX. Select the checkpoint supported by your exact accelerator and inference runtime, following Meta's checkpoint guide and platform-specific deployment instructions. GGUF repositories provide a different packaging and quantization path for llama.cpp. Runtime compatibility and modality support must be checked for the exact files you select.

These are weight footprints, not complete serving-memory budgets. Leave room for cache, activations, the operating system, and runtime allocations. A lower-bit conversion may reduce storage while changing quality or throughput; test those tradeoffs on your actual prompts.

How to Download Muse Spark Open

Start with Python 3.10 or later, persistent storage, and the official NVFP4 repository. Review its license and complete any access requirements shown to your account. The BF16 checkpoint is a separate repository. The download below saves a local copy; the later Docker example uses a persistent Hugging Face cache, so it may download another copy unless you explicitly configure local-path loading.

Download the NVFP4 checkpoint to persistent local storage.

python -m pip install -U huggingface_hub
hf auth login
hf download meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --local-dir ./Muse-Spark-Open-NVFP4

Plan for hundreds of gigabytes of transfer and additional space for temporary files. If the download returns HTTP 401/403, confirm repository access and authentication before changing the inference stack. Avoid copying a token into published shell history or screenshots.

How to Deploy Muse Spark Open with vLLM

Use vLLM when you want an OpenAI-compatible endpoint on a supported accelerator server. Meta now documents a four-B300 NVFP4 starting configuration. The launch example below follows that topology, lowers the served context to 128K, and leaves optional speculative decoding disabled for the first validation run.

Check the prerequisites

Prepare a Linux server with four compatible B300 GPUs, the matching NVIDIA driver, Docker, NVIDIA Container Toolkit, and enough persistent storage for weights and cache. Fix GPU visibility inside containers before downloading the model. Run the Bash examples in a Bash shell.

Check the host, Docker, and GPU access inside a CUDA container.

nvidia-smi
docker --version
docker run --rm --gpus all nvidia/cuda:12.8.1-base-ubuntu22.04 nvidia-smi

Set the Hugging Face token

If repository access requires authentication, pass your token through the environment. The following prompt hides the token while you type; do not paste credentials into shared commands or published logs.

Read a Hugging Face token without echoing it.

read -rsp "Hugging Face token: " HF_TOKEN
echo
export HF_TOKEN

Pull the model-specific container

Pull the Muse Spark Open vLLM image.

docker pull vllm/vllm-openai:muse-spark-open

Use the current Meta vLLM configuration with a Muse-capable build. A generic older image may not recognize the architecture or parser. Save the image digest after a successful test so later deployments use the same build.

Launch the NVFP4 server

Launch a four-GPU vLLM server with a 128K serving limit.

docker run --rm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -e HF_TOKEN \
  -e VLLM_USE_RUST_FRONTEND=1 \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  vllm/vllm-openai:muse-spark-open \
  meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --max-model-len 131072 \
  --reasoning-parser muse \
  --tool-call-parser muse \
  --enable-auto-tool-choice

This is a starting configuration for the specified topology, not a promise that any four GPUs can serve the checkpoint. Let the first download, model loading, and compilation finish. Preserve the cache between restarts. This command fetches from the repository into the mounted cache; it does not automatically reuse the separate --local-dir download above.

Test the local endpoint

List the served model and request a short answer.

curl http://localhost:8000/v1/models
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model":"meta-models/Muse-Spark-Open-816B-A42B-NVFP4",
    "messages":[{"role":"user","content":"Explain tensor parallelism in two sentences."}],
    "reasoning_effort":"low",
    "max_tokens":2048
  }'

Use the identifier returned by /v1/models if it differs. Check both the final answer and finish_reason: reaching the token limit can cut off reasoning before a useful answer appears. Keep localhost binding for initial tests and add access control before remote use.

Connect a Python application

Install the Python client used by the example.

python -m pip install -U openai

Call the local vLLM endpoint from Python.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
    timeout=3600,
)
model_id = client.models.list().data[0].id
response = client.chat.completions.create(
    model=model_id,
    messages=[{"role": "user", "content": "Write a small Python input validator."}],
    max_tokens=2048,
    extra_body={"reasoning_effort": "low"},
)
print(response.choices[0].message.content)
print("finish_reason:", response.choices[0].finish_reason)

EMPTY is only a placeholder for a local server without API-key enforcement. Use the configured credential if you enable authentication. Record the served model ID and request settings when integrating agents.

How to Deploy Muse Spark Open with SGLang

SGLang is another supported native-weight serving option. Its Muse Spark Open cookbook provides hardware-specific configuration and tuning. Select the NVIDIA image for B300; AMD and Intel use different images and recipes.

Prepare the model-specific image

Pull the NVIDIA SGLang image for Muse Spark Open.

docker pull lmsysorg/sglang:dev-muse-spark-open

Start the four-B300 NVFP4 server

The following basic launch adapts the native NVFP4 settings, with speculative decoding and hierarchical caching omitted. Enable those features only after the base endpoint works and memory headroom is measured.

Start SGLang with four-way tensor and expert parallelism.

docker run --rm --gpus all --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -e HF_TOKEN \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  --entrypoint sglang \
  lmsysorg/sglang:dev-muse-spark-open serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --tp-size 4 --ep-size 4 \
  --quantization modelopt_fp4 \
  --moe-runner-backend flashinfer_trtllm \
  --mem-fraction-static 0.8 \
  --context-length 131072 \
  --enable-layernorm-sp \
  --enable-megamoe-for-layernorm-sp \
  --reasoning-parser muse \
  --tool-call-parser muse \
  --host 0.0.0.0 --port 30000

Verify the SGLang endpoint

Check the model list before reusing the chat request on port 30000.

curl http://localhost:30000/v1/models

Send the same short chat test used for vLLM to port 30000, using the model ID returned here. Recheck final-answer completion and a harmless tool call before running a coding agent.

How to Run Muse Spark Open with llama.cpp

The ggml-org GGUF repository provides a separate conversion route. Review the chosen files, available RAM/VRAM, and current runtime support before starting a download. GGUF availability does not make an 816B model fit an ordinary laptop.

Install a current llama.cpp build

Install the Llama app binary on a supported macOS or Linux host.

curl -LsSf https://llama.app/install.sh | sh

Review installation scripts before running them and use the platform-specific instructions if your environment blocks shell installers. The commands below use the current llama application interface; older binaries named llama-server use a different CLI.

Start the GGUF server

Serve the Q4_K_S GGUF with a fixed API alias and a 32K initial context.

llama serve \
  -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q4_K_S \
  -a muse-spark-open \
  -c 32768 \
  -np 1 \
  --chat-template-kwargs '{"reasoning_effort":"low"}'

Confirm the GGUF server is reachable on its local endpoint.

curl http://localhost:8080/v1/models

Use muse-spark-open as the request model alias. Keep the machine on a trusted network while testing, and configure binding and authentication before sharing access. Confirm vision, audio, and tool behavior for the selected conversion rather than assuming feature parity with native weights. Use Ollama only with a verified Muse Spark Open model package and a compatible version.

How to Run Muse Spark Open on Multiple Mac Studios with MLX

Use two M5 Ultra Mac Studios with 512 GB of unified memory each, or four with 256 GB each. This is a distributed setup, not an ordinary MacBook installation.

The official MLX deployment instructions require macOS 26.2 or later, MLX 0.30.1 or later, RDMA enabled on every Mac, direct Thunderbolt 5 links between every pair of Macs, and passwordless SSH from the primary host. Each host needs disk space for the full NVFP4 checkpoint.

Install mlx-lm on every host. Use mlx.distributed_config and the MLX distributed guide to generate hostfile.json with each machine’s Thunderbolt IP addresses and RDMA devices. Check SSH access and the RDMA configuration before launching.

Start the distributed OpenAI-compatible MLX server from the primary Mac.

mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
  mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080

Verify the endpoint from the primary Mac with curl http://localhost:8080/v1/models. Reuse the returned model identifier for a short chat request. Keep the service on a trusted network and add authentication before sharing access.

Which Muse Spark Open Checkpoint Should You Choose?

Start with NVFP4 on a supported Blackwell server when its memory and interconnect match the native recipe. Use BF16 when the full-precision reference is required and you have the capacity. Choose a specific GGUF conversion for llama.cpp only after checking its actual file size and output quality. For the documented multi-Mac route, use the MLX-compatible NVFP4 checkpoint.

Avoid choosing solely by the smallest download. Compare your own task success, response completeness, latency, and memory use. Keep the exact repository revision and runtime build in the deployment record so results can be reproduced.

Muse Spark Open: vLLM vs SGLang vs llama.cpp vs MLX

Choose a runtime using your existing hardware and the model-specific recipe. vLLM suits supported accelerator servers; MLX covers the documented multi-Mac setup. Meta also documents llama.cpp with GGUF checkpoints. Validate the chosen checkpoint and backend before estimating throughput or switching runtimes.

How Do You Optimize a Local Muse Spark Open Deployment?

Increase context gradually

Use a bounded context for the first launch and raise it only when the workload needs more history. Monitor prefill time and memory as input length grows. A model supporting a 1M-token window does not guarantee that every hardware configuration can reserve that capacity at useful concurrency.

Tune one feature at a time

Measure the base server before enabling prefix caching, speculative decoding, or advanced distributed layouts. Repeat the same prompts and traffic pattern after each change. Keep changes that improve completed-task latency or throughput without harming output quality.

Keep storage and runtime versions predictable

Place weights and cache on persistent fast storage, separate cold startup from warm requests, and pin validated image digests. Record accelerator type, checkpoint revision, runtime build, parser settings, context limit, and concurrency with the measurements.

Monitor completed work

Check correctness before speed. Start with a short text request and confirm that the server returns a complete answer. Then test a harmless tool call and an image request if the chosen runtime supports them. Record software versions, GPU count, checkpoint, input length, output budget, and concurrency for every local measurement.

Increase prompt length and concurrency gradually, watching first-token latency, completed responses, and peak memory. These are local validation measurements; this guide does not claim measured throughput or hardware break-even figures.

How Can You Use Muse Wonder API in CometAPI Instead of Self-Hosting?

If a multi-GPU server or a distributed Mac setup is impractical, you can use Muse Wonder API in CometAPI as a hosted Meta-model alternative. This is not an API endpoint for the Muse Spark Open checkpoint: the gateway model ID is muse-wonder, while Muse Spark Open refers to separately released downloadable weights. Do not transfer Muse Spark Open benchmark results or exact model-feature claims to Muse Wonder.

Endpoint, model identity, and authentication

Use the HTTPS base URL https://api.cometapi.com/v1, POST /chat/completions, Bearer authentication, and the model identifier muse-wonder. See the CometAPI integration documentation for authentication details.

export COMETAPI_KEY="your_cometapi_api_key"
 
curl -X POST "https://api.cometapi.com/v1/chat/completions" \
  -H "Authorization: Bearer $COMETAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "muse-wonder",
    "messages": [{"role":"user","content":"Explain expert parallelism briefly."}],
    "stream": false
  }'

A successful response should include an answer in choices[0].message.content. The request illustrates the documented CometAPI route; it is not evidence that Wonder and Spark Open are the same model.

Python client example

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["COMETAPI_KEY"],
    base_url="https://api.cometapi.com/v1",
    timeout=120.0,
)
result = client.chat.completions.create(
    model="muse-wonder",
    messages=[{"role": "user", "content": "Summarize the deployment tradeoffs."}],
)
print(result.choices[0].message.content)

What inputs and pricing does the alternative support?

Meta documents text, image, video, audio, and PDF input for native Muse Wonder, with text output. Gateway support for each media format and message schema must be validated separately; the examples above deliberately start with text. The CometAPI Muse Wonder API guide provides the following published rate snapshot: $3.20 per million uncached input tokens and $16.00 per million output tokens. It does not establish a CometAPI cached-input tariff. Check the rate applied to your account before production use.

At those quoted rates, 100,000 uncached input tokens plus 50,000 output tokens would cost approximately $1.12 in model-token charges (0.1 × $3.20 + 0.05 × $16.00), excluding other possible fees. By contrast, local hosting costs depend on GPU acquisition or rental, power, utilization, storage, engineering, and maintenance. This is a pricing example, not a measured cost-per-task benchmark.

Self-hosting offers control over data residency, runtime configuration and potentially predictable high utilization. Hosted APIs avoid buying accelerators, maintaining expert-parallel clusters, and engineering fault tolerance. The break-even point is workload-dependent: a lightly used internal assistant usually has different economics from a heavily utilized coding agent fleet.

Common Muse Spark Open Local Deployment Problems

Separate startup failures from request failures

For startup failures, check repository access, GPU visibility, architecture support, and memory allocation in that order. If the server is ready but requests fail, inspect the model ID, request format, parser, and output budget. Save the runtime version and the first relevant error instead of retrying with several unrelated flag changes.

Conclusion

If you have a supported Blackwell server, begin with NVFP4 and an official vLLM deployment recipe, validate a modest serving context, and scale parallelism after collecting your own latency and task-success measurements. If you are working entirely in the Apple ecosystem, the supported desktop story is a multi-M5 Ultra Mac Studio MLX deployment, not a conventional laptop install. For teams without this infrastructure, use a hosted inference provider while keeping model identity and price assumptions explicit.

FAQ

How can I restart a Muse Spark Open download after an interruption?

Rerun hf download with the same repository and local directory. Keep the downloaded files and cache so the client can reuse completed content. Confirm that authentication still works and enough disk space remains before retrying; deleting the entire directory can force another large transfer.

What happens to Muse Spark Open conversations after a server restart?

Treat the HTTP inference endpoint as stateless unless your application explicitly persists conversation history. Save the messages and tool results needed to reconstruct the conversation, and send them again after restart. A persistent model-download cache preserves weight files, not your application’s conversations.

How do I keep Muse Spark Open reasoning settings consistent?

Set the supported reasoning effort explicitly in the client request instead of relying on defaults. Record that value with the model revision, chat template, and runtime version when comparing responses. If behavior changes after an upgrade, check those settings before attributing the difference to the checkpoint.

How can several applications share a local Muse Spark Open endpoint?

Point each client at the same OpenAI-compatible base URL and the model identifier returned by /v1/models. Add authentication, request limits, and monitoring before sharing it with more users. Test simultaneous requests and define queueing or timeout behavior before treating a successful single request as a capacity guarantee.