Reading Ollama discord channel, I notice many people want to self-host their chatGPT with Docker and don’t know how to do it. Here’s how to host the whole stack with docker compose. Here’s my docker-compose.yml including the mitmproxy from the previous article. version: "3" services: ollama: build: ollama user: 1001:1001 environment: - OLLAMA_HOST=0.0.0.0 - OLLAMA_DEBUG=1 - OLLAMA_KEEP_ALIVE=60m volumes: - /etc/localtime:/etc/localtime:ro - ollama_models:/home/ollama/.ollama/models mitmproxy: image: mitmproxy/mitmproxy command: mitmweb --web-host 0.0.0.0 --web-port 8080 --mode reverse:http://ollama:11434@11434 --verbose --anticache --anticomp depends_on: - ollama labels: - "traefik.enable=true" # ollama endpoint - "traefik.http.routers.ollama.rule=Host(`llm.example.com`)" - "traefik.http.routers.ollama.tls=true" - "traefik.http.routers.ollama.entrypoints=websecure" - "traefik.http.routers.ollama.tls.certresolver=le" - "traefik.http.routers.ollama.service=ollama" - "traefik.http.services.ollama.loadbalancer.server.port=11434" - "traefik.http.services.ollama.loadbalancer.server.scheme=http" # mitmweb endpoint - "traefik.http.routers.ollama-mitm.rule=Host(`inspector.example.com`)" - "traefik.http.routers.ollama-mitm.tls=true" - "traefik.http.routers.ollama-mitm.entrypoints=websecure" - "traefik.http.routers.ollama-mitm.tls.certresolver=le" - "traefik.http.routers.ollama-mitm.service=ollama-mitm" - "traefik.http.services.ollama-mitm.loadbalancer.server.port=8080" - "traefik.http.services.ollama-mitm.loadbalancer.server.scheme=http" - "traefik.http.middlewares.ollama-mitm-headers.headers.customrequestheaders.Host=0.0.0.0" - "traefik.http.middlewares.ollama-mitm-headers.headers.customrequestheaders.Origin=" open-webui: build: context: . args: OLLAMA_API_BASE_URL: '/ollama/api' dockerfile: Dockerfile image: ghcr.io/open-webui/open-webui:main volumes: - /etc/localtime:/etc/localtime:ro - open-webui:/app/backend/data depends_on: environment: - 'OLLAMA_API_BASE_URL=http://mitmproxy:11434/api' - 'WEBUI_SECRET_KEY=' labels: - "traefik.enable=true" - "traefik.http.routers.open-webui.rule=Host(`chatgpt.example.com`)" - "traefik.http.routers.open-webui.tls=true" - "traefik.http.routers.open-webui.entrypoints=websecure" - "traefik.http.routers.open-webui.tls.certresolver=le" - "traefik.http.routers.open-webui.service=open-webui" - "traefik.http.services.open-webui.loadbalancer.server.port=8080" - "traefik.http.services.open-webui.loadbalancer.server.scheme=http" volumes: ollama_models: open-webui: This exposes 3 different endpoints: ...

Sometimes you connect a new tool to one of your servers and it doesn’t work as expected. You are sure you follow the documentation or tutorials but you don’t get the expected results. Before you throw away everything, you should check what’s actually going on between the 2 applications. And if none of them supports logging requests and responses, you can use mitmproxy for troubleshooting. As the name imply (MITM = Man In the Middle), mitmproxy sits between both applications and intercepts all the traffic. You can use it to log the traffic but also modify the content of the requests and/or responses? On the fly. I will not cover that use-case here. ...

Ollama I have recently started to use Ollama and I was unimpressed by some models as they did not follow instructions, especially in their output format. I knew about model system prompt but I thought it was fixed in the model. Then I found out you could change the system prompt at run time with the /set system command and immediately, most models responded as expected. That was so much better! ...

I have just bought a mini PC based on Intel N100 CPU. Initially, I was going to buy another Raspberry PI or a used “TinyMiniMicro” PC, but I decided to have a look at the current mini PC offering. I am glad I did. On a major Chinese reseller website, I saw a lot of similar products with the Intel N100 CPU so I had a look at reviews (here here) and boy, this thing is powerful (for its size). I’ll talk about the mini PC in an other post. ...

By default, resources available to containers are not limited. However, sometimes, you want to make sure a container is not going to use too much processing power or memory. To achieve such a thing, in the docker-compose.yml file, add the following sections to the service you want to restrict: deploy: resources: limits: cpus: "1.0" memory: 100M memswap_limit: 100M This will effectively limit the container to use at most one CPU and 100 megabytes of memory. ...

If you cannot upgrade your OpenSSH client and/or server to fix CVE-2023-48795, also known as the Terrapin attack, the way to mitigate it is to disable the vulnerable ciphers as Red Hat explains very well. If you have a recent OpenSSH version, you can disable the the ciphers by adding “-” before them in the Ciphers and MACs options. This works for both the ssh client config (/etc/ssh/ssh_config by default) and the ssh server config (/etc/ssh/sshd_config). ...

I wanted to help a friend learning English who has trouble learning new vocabulary. She often gets new list of words at school and it’s difficult for her to know how to use them, or remember what they mean. She usually gets one exercise about the topic where she must fill blanks with words from a list. Why not use generative AI for that? I could not achieve good results using a single large prompt, so I decided to explicitly break it into different steps and refer to the whole process later, with “OK” results. ...

I’m a personal user of Contabo’s cloud services, and I’ve been delighted with them. They offer a wide range of services to choose from, including VPS, dedicated servers, and cloud storage. I’m currently using a VPS to host my personal website and email, and I have also used their Storage VPS and object storage in the past. I have had no issue with my VPS over the years. I’ve also been impressed with Contabo’s customer support. They’ve always been quick to respond to my questions. ...

I ran the same prompt using many samplers at different steps counts to evaluate which one(s) give a decent quality at a low step count. I have not used the “restore faces” option. Here are my observations related to image quality (artifacts) and convergence. Quality at lower steps At 10 steps, a few samplers are unusable: DPM++ 2M and its variants, DDIM. At 15 steps, all samplers are OK except DPM++ 2M SDE and its Karras variant are unusable. ...

And voilà! SDXL 1.0 is out. After tinkering a bit, I think it’s working pretty well. As with SDXL 0.9, I must use both base and refiner models to get good pictures, but they are of excellent quality. Use the pipeline from ComfyUI and put the models at the right place: https://comfyanonymous.github.io/ComfyUI_examples/sdxl/ Note that it’s really slow with an AMD Radeon RX 6700 XT, especially because of the 2 models. A few links: ...