: Significant number of non-HTTP requests hitting my site I'm seeing a significant number of non-HTTP requests hitting a site I just launched. They show up in the server (nginx) logs as non-ASCII

@Gloria169

I've worked some pretty large sites (1bn pv/day on the largest), approx 1-2% of their traffic was random exploit probing/spiders/crawlers.


Classic path manipulation stuff like ......cmd etc.
More sophisticated buffer-overruns in POST and/or GET looked similar to those in your logs
Plain screwed-up home-rolled crawlers that mis-encoded or omitted key aspects of the HTTP protocol.
glitched customer connections garbling transmitted data.


Seems that it's plausible that it could be a buffer overrun attack - do you see other instances of the same request pattern? Quite plausible to see screwed up clients and connections too. Switching on logging for other W3C variables like user agent etc. may also give you more clues. e.g. the absence of regular headers like accepts, method etc.

We generally:


made sure our servers were patched regularly just in case ;-)
used tools like UrlScan (we were on IIS) to throw out malformed requests early in the request pipeline.
capped our POST and GET request sizes to prevent some of the more crazy DDoS attacks.
Pawed through all non-2xx/3xx traffic from the logs on a regular basis watching for new patterns. Believe it or not, Excel is an excellent tool for quickly scanning and slicing and dicing data.


While some actions are IIS specific, I've no doubt that nginx has similar tools. In generally there is now so much noise on the internet from random bots, I've learned to just be vigilant and look for emerging new trends.

Vote Up Vote Down