6 posts tagged with "DevOps"

Cloudflare injects cf-ipcity into incoming requests, and for visitors in Vietnam the value is Hồ Chí Minh — with diacritics, which means non-ASCII. Our .NET service forwarded every incoming header verbatim onto its outgoing calls, so that value landed in HttpClient. The surprise is that Headers.Add does not throw, and TryAddWithoutValidation returns true; everything only blows up at SendAsync with HttpRequestException: Request headers must contain only ASCII characters, and not a single byte leaves the process. The bug is neither Cloudflare's nor .NET's — it is in an application that forwards every header unconditionally.

onnxruntime-node ships prebuilt binaries built against glibc, and libonnxruntime.so.1 carries a DT_NEEDED entry pointing directly at ld-linux-x86-64.so.2 — glibc's own dynamic loader. Alpine uses musl, where that file does not exist, so dlopen fails at the library-loading step. Installing libc6-compat or gcompat does not fix it: they provide the loader file so you clear the first error, then die on the second one, __vsnprintf_chk: symbol not found. The practical answer is to change the base image to node:22-bookworm-slim, paying about 89 MB more for something that actually runs.

When a domain is proxied through Cloudflare (the orange cloud) with SSL mode Full (strict), ACME HTTP-01 cannot work: Let's Encrypt requests http://domain/.well-known/acme-challenge/... on port 80, Cloudflare receives it and calls back to your origin over HTTPS:443 — where Traefik's challenge handler does not live. You need a certificate to get through Cloudflare, and you need to get through Cloudflare to obtain a certificate. You can work around the first issuance by temporarily switching the cloud to grey, but the automatic renewal around day 60 will fail silently and every site will expire at the same time. The way out is DNS-01: validation through a TXT record via the Cloudflare API, with no request ever touching the origin. The price is that certificates can only be issued for zones your API token can see.

In January 2025 I was handed a product to initialise — the first time in my career — a multi-tenant CRM platform whose repository was completely empty when I took it on. For nearly its first year it lived in demo mode, shown to customers across many different industries, until late 2025 when a healthcare client signed and it moved into real delivery. Twenty months later it is 18 microservices on ABP Framework / .NET 9 and 13 Angular libraries, built by a team that grew to 25 people. This is not a technical article but a career story told chronologically, and it includes the export screen that forgot its permission checks, the two weeks of Angular work I had to revert, the three months mid-year when I was moved to a different loyalty project and committed nothing, and a migration so boring that nobody noticed it had happened. It also covers the EAV architecture that lets customers configure their own data fields from the portal, along with the months I spent wrestling with its performance and that of the dynamic filter sitting on top of it, the stretch where I mentored four interns for the first time, and how the way I write code shifted from typing everything by hand to ChatGPT, then Cursor, then Claude.

An ASP.NET Core 9 container sat at 987 MiB against a 1 GiB ceiling (96%) after two hours without receiving a single request. This was not a memory leak: the number was absolutely flat across repeated measurements, which is the signature of retained heap rather than a leak. The cause was Server GC — the ASP.NET Core default, which creates one heap per CPU core and releases very little memory back to the operating system. Setting DOTNET_gcServer=0 and DOTNET_GCConserveMemory=5 brought anonymous memory down from 822 MB to 280 MB (−65%), measured as stable after 30 minutes. The trade-off: Workstation GC gives lower throughput under heavy load on many-core machines.