17.06.13

(Very Very Simple) Bash Script to Delete Old Log Files

Posted in News, Random Thoughts, Tech at 3:25 pm by Liv

So here’s the deal: you have an application running on a Linux box which generates log files. And these files keep mounting up and grow and grow and grow in both number and size and you have start considering cleaning up old logs which you don’t use anymore. I know about logrotate, it’s an awesome tool — assuming that:

• your application knows to close the current log file on a signal (e.g. SIGHUP) — so logrotate can “rotate” it
• or your application is ok with being “hung” for the period of time it takes logrotate to take a snapshot of the file and then truncate the existing one
• and also your application doesn’t have an in-built log rotation mechanism itself

Now, if you happen to have a server application running which is doing a lot of extensive work, you know damn well that you cannot afford to freeze that even if it is for milliseconds, as that would deteriorate the experience to the clients “consuming” your server app. And if it so happens that the amount of logging you do is in the order of a few hundred Mb a day, then you know as well that this won’t take just a couple of milliseconds. Also, if you use something like Java — which is my case and what triggered in the end this blog post, unlikely you’ll be able to hook into SIGHUP or similar; even more, if you already use one of the Log4J’s built-in mechanisms to auto-rotate the log files (be it on size, date etc), then again, you cannot use logrotate unfortunately. However, you will still have each day a new log file being rotated (if you decided to have Log4J rotate on a daily basis) and these files keep mounting up and taking disk space; while you might keep an eye on the log files from the last week, unlikely I’d say in most cases you’d do the same with log files created 3 months ago — so there’s a lot of wasted disk space there!

Read the rest of this entry »

Permalink Comments

Disclaimer

08.01.13

CDN — The Basics — Part 2

Posted in Blogroll, Random Thoughts, Tech at 4:07 pm by Liv

In the previous post, “CDN — The Basics — Part 1″, we got as far as looking at the situation where a load ballancer is employed in front of a set of identical web servers, and the DNS is handled externally by a dedicated DNS server, potentially in a different cloud/region/data center. As it has been the case so far with this topic, we’ll look at how this structure can be improved and look at incremental changes that can be made to increase reliability (and as such uptime). We will get to the CDN bit too eventually, don’t worry

In the last showcased diagram in the previous post in these series, we have looked at scaling the web app horizontally by adding more servers and have a load ballancer in front of them. This allows us to scale nearly infinitely the web application by adding more webservers. However, since all requests have to go through the load ballancer, this creates a bottleneck — as much as a single point of failure. (Neither of those are desirable, needless to point out!) So to overcome that, there are multiple ways of dealing with this — but each one of these methods relies on the same idea: add more load ballancers in the setup!

Read the rest of this entry »

Permalink Comments

Disclaimer

03.01.13

CDN — The Basics — Part 1

Posted in Blogroll, Random Thoughts, Tech at 5:40 pm by Liv

I wish I could put a “1 out of N” at the end of this post title, however, I have no ideas how far this will go unfortunately; I do however realise that the more I go into details about this subject the more there is to be said about it. And what I have realized even more recently is that while the general principle is known about a CDN (“it makes your website download faster”?!?!?!) all these details that can shave off easily miliseconds off your CDN setup are not known or are not understood. As such, this post wants to be the first in a tutorial about CDN (Content Delivery Networks) and about various components around them that often get forgotten when planning and setting up a CDN.

It has to be said from the beginning that I am not the definitive authority in this! Part of the reason why I envisage this to be an ongoing tutorial is that I’m pretty sure that I will keep finding out more and more about this subject in time — and as I do learn more I’ll publish more here. It is also worth mentioning that because of this, there are chances I will be missing out certain aspects too — and if you are aware of these, I would so love to hear about these! (Mainly so I can improve on my knowledge in this area, but rest assured that I will publish those facts on the blog too.) So drop me a line and let me know if I missed anything.

Read the rest of this entry »

Permalink 1 Comment

Disclaimer

09.08.12

Application Monitoring and Management using Sun’s JMX/HTML Interface

Posted in Blogroll, News, Tech at 11:48 pm by Liv

This is something I wanted to write about a while back – while there are articles on the net about using JMX in a Java application to keep an eye on how it ticks, or manage its running cycle, I think there is still a large number of users out there who are somewhat reluctant to use this part of the JDK to achieve this in their Java applications. Reasons I think are quite varied, and they span from the large availability of enterprise-wide monitoring tools out there (so much easier to outsource this part to another company so you can concentrate on building the application) to the fact that the JMX Console that gets shipped with the JDK is not the most scriptable to use (it’s a Swing app after all!) or the most friendly. (In particular, it uses RMI and trying to configure a firewall to allow access via JMX Console to your infrastructure proves a pain, since it uses dynamic ports.)

Few know though that there is a little package from Sun (ahem, Oracle I mean!) who provides a HTML layer for inspecting your MBean’s and invoking actions on them. While this might not sound like much (in fact, I am pretty sure Oracle’s JDMK is not the only HTML/JMX interface, so it’s not a big deal after all that Oracle provides one), it can have some (sweet) implications in your app management and monitoring infrastructure:

• Read the rest of this entry »

Permalink Comments

Disclaimer

31.10.11

Configure autofs auto.smb on Debian Linux

Posted in Blogroll, Tech at 2:58 pm by Liv

Yet another post from my old website brought back to life — just as with the previous such posts, I hope this still stands. Nowadays I’ve moved most of my “infrastructure” to Amazon, I don’t have to use this anymore, but I remember at the time having to do a bit of research into this to get this automount procedure going. So hopefully is not that out of date and/or misleading!

If you’re using automounter (autofs) on Debian Linux and you have tried to configure it so it automatically mounts Windows shares automatically via auto.smb, you probably got stuck like myself at the first step. As you might have noticed, the auto.smb file actually is a shell script (the #!/bin/sh shebang in the beginning gives it away and the file permissions) which doesn’t seem to do any mounting! It seems to work though (in certain configurations) if you do a ls /mountpoint/SAMBA_NAME and it lists all the shares on the machine SAMBA_NAME. But from there on you should still use smbmount to mount the share manually.
Read the rest of this entry »

Permalink 2 Comments

Disclaimer