Run QWEN 3.6 / GPTEL / AIDER on your Apple Silicon

If you run LLMs locally on Apple Silicon, you already know that mlx-lm provides great performance thanks to unified memory. However, interacting with it through random bash scripts or kicking off commands manually in the CLI gets tedious. I wanted to shift away from brittle automation hacks and treat the model infrastructure as code, integrating the entire ecosystem directly inside Emacs.

The goal was to run a quantized Qwen 3.6 (35B-A3B) and use it as a stable backend for both gptel (for quick, context-aware chats) and Aider (for multi-file repository refactoring).

Along the way, I had to resolve some Python environment mismatches and handle local model parsing limitations inside Emacs, but the end result is a streamlined, programmatic workflow. Here is how it is put together.

Privacy and Control: Keeping Data in the Room

Beyond the architectural advantages, there is a fundamental data privacy argument for this setup. When you rely on commercial cloud endpoints, your proprietary codebases, refactoring patterns, and internal logic are continuously ingested by remote servers. Running a local pipeline ensures that your context remains entirely on your machine. For anyone who objects to feeding proprietary data into commercial models to train corporate assets, treating a local model as a private service layer is the only real way to regain total structural control over your code.

Moving mlx-lm to a Programmatic Python Configuration

Initially, configuring or embedding mlx_lm.server dynamically meant wrapping a CLI command or invoking it via a clunky shell subprocess call in Python. It felt like an external execution step rather than an application layer. I wanted to handle mlx-lm as a first-class citizen in my codebase—importing it, configuring it with native data structures, and running it directly in Python.

The pull request is here (#1213) and I opened a discussion about its utility here (#1212) if you want to check it out. The patch refactors the server initialization by exposing run_server and MLXServerConfig cleanly to the public API.

Thanks to this change, you no longer need to invoke the server as an external black box. You can configure and boot the server natively inside a Python script:

from mlx_lm.server import run_server, MLXServerConfig

# MODEL="mlx-community/Qwen2.5-Coder-32B-Instruct-4bit"
MODEL="mlx-community/Qwen3.6-35B-A3B-4bit"

config = MLXServerConfig(
    model=MODEL,
    port=8912,
    max_tokens=32768
)

# config.chat_template_args = {}
run_server(config)

By turning the execution into a programmatic configuration block, mlx-lm can be integrated into local tooling or automation pipelines in a clean, Pythonic way.

Dependency Isolation with pipx

With the server running as a Python service, the next step was setting up the client tools. If you are working on cutting-edge environments (like Python 3.14 setups), installing large terminal tools like Aider frequently drags you into dependency mismatches because core math binaries like numpy fail to compile their wheels cleanly.

To keep things stable, I used pipx to force Aider’s installation inside an isolated virtual environment running Python 3.12. This leaves the executable globally accessible from the system shell without breaking or cluttering the host system:

pipx install aider-chat --python python3.12

Integrating Everything Into Emacs (init.el)

The final mile was configuring Emacs to talk to the local server for both quick buffer chats and deep workspace editing.

Here is the exact portion of my init.el that links gptel and Aider to the local server application:

(use-package gptel
  :ensure t
  :config
  (setq-default gptel-backend
                (gptel-make-openai "MLX-Local"
                  :host "localhost:8912"
		  :protocol "http"
                  :endpoint "/v1/chat/completions"
                  :stream t
                  :models '("mlx-community/Qwen3.6-35B-A3B-4bit")))

  (setq-default gptel-model "mlx-community/Qwen3.6-35B-A3B-4bit")
  (add-hook 'gptel-post-response-functions 'gptel-end-of-response)
  (global-set-key (kbd "C-c g") 'gptel-send))

(use-package aider
  :ensure t
  :bind
  (("C-c a a" . aider-run-aider)
   ("C-c a f" . aider-add-current-file)
   ("C-c a r" . aider-read-only-add-current-file)
   ("C-c a d" . aider-drop-current-file)
   ("C-c a c" . aider-code-change))
  :config

  (setq aider-backend 'comint
        aider-args '("--no-analytics" "--no-suggest-shell-commands" "--no-show-model-warnings")))

To ensure Aider targets the correct model string by default when spinning up, a minimal global configuration file in your home directory (~/.aider.conf.yml) wraps it up cleanly:

openai-api-base: http://localhost:8912/v1
openai-api-key: not-needed
model: openai/mlx-community/Qwen3.6-35B-A3B-4bit
edit-format: diff
stream: false

Conclusion

Shifting from loose terminal invocations to managing a local LLM as a programmatic service layer results in a much more stable environment. Exposing the server initialization directly in mlx-lm allows for clean configuration scripts, while a sanitized Emacs sub-process environment enables a private, 35-billion parameter copilot workflow that runs entirely on your local machine.