Perl One-Liner of the Month: The Adventure of the Spicy Blonde
By Ben Okopnik
It was a blonde. A blonde to make a bishop kick a hole in a stained glass window.[ Author's note: These events... well, something similar... or maybe not, anyway something or other actually happened. Names, genders, actions, phrasing, situations, environment, weather, and possibly even the planet of occurrence have all been changed in order to punish the innocent and protect the guilty. Even the tea was Guangdong Gunpowder instead of Earl Grey, and wasn't served nearly hot enough. ]
-- Raymond Chandler
Frink staggered into Woomert's office, somewhat pop-eyed.
- "Did you see... she's sitting out there, Woomert! She's, well, uh... tall, and..." His hands were waving wildly but still describing arcs with enough curvature that Woomert raised an eyebrow and glanced toward the door to the living room.- "Well, Frink - that's quite a description. First, take a deep breath... that's it, again... one more time... good. Now, what's it all about?"
Frink cleared his throat.- "Right. Well, for one thing, she's really upset; her Web server died, her hard drive crashed, her SCSI controller bit the dust - before you say anything, though, the hardware's been fixed already. The problem is that they still can't get the server up - it keeps dying on startup - and she says that someone's been stealing her domain names! That's all I got, though: she's, uh, a little distracting, especially when she's angry... you'll see what I mean when you talk to her."
Woomert steepled his hands and looked at Frink for a long moment.- "When you talk to her and I observe, you mean. I meant it when I said that I wanted you to be my partner, Frink; you'll need to learn all the aspects of being an Internet Detective, and this sounds like the perfect opportunity."
- "But... but..." Frink looked thoroughly lost. "Me? You want me to talk to her? But what if I mess it all up?"Woomert stood up and walked across the room, and stood by the door, obviously waiting for Frink to go through first.
- "If you mess it up, it'll be a horrible tragedy and will destroy both our reputations forever... riiight! Frink, what I'm asking you to do is your best; if any problems come up, I'll be there to help. Buck up, lad; I have faith in you."Frink looked at him for a few seconds, then took a deep breath, squared his shoulders and nodded. In all the time that the two had known each other, Woomert had never known the younger man to look so serious, or so intent on doing the right thing. He stepped into the living room just after Frink, and quietly closed the door.
- "Miss Bombshell, I'd like to introduce my senior partner, Woomert Foonly. Woomert, Miss Auburn Bombshell."Woomert's first thought was that Frink had not exaggerated. The woman in front of him, dressed in a red leather ultra-miniskirt and a white silk blouse that was at least one size too small, was quite distracting - stunning, actually. Woomert's eyes briefly swept over her generous curves and came to rest on her eyes, blue as sapphires of the first water... and, just like those sapphires, full of fire. She had been paying only cursory attention to Frink, and now turned the full force of her personality on Woomert.
- "Well, well - it's the famous Woomert Foonly! I've heard a lot about you. Well, now that you're here, we should be able to get something accomplished. You see..."- "Just a moment, Ms. Bombshell." Woomert stood easily, hands in his pockets and leaning against the wall. "You seem to be laboring under a misconception; I am indeed the senior partner here, but I've recently" - there was no need to specify just how recently - "begun acting in a largely advisory capacity, while my partner, Frink Oooblick, does most of the work." He smiled slightly. "Rank hath its privileges, time goes on. Fear not: Frink is quite capable, and can call on me if the need should arise - although I don't expect that to happen."
- "But... I came here particularly for your services!" The blonde looked even more angry than before, which did not seem possible; Frink expected to see lightning strike at any moment. "I demand..."- "Ms. Bombshell." There was obvious frost in Woomert's voice. "If you wish to deal with us, then you need to know this: I and my partner behave in a civil manner toward our clients, and we require the same in return. I have stated what is on offer; it is up to you to accept or find help elsewhere. Being offensive will simply limit your options to the latter."
- "There's nowhere else to look - I've tried everywhere!"- "Then I suggest you abide by the rules I've stated." Woomert crossed to the pantry and began filling the kettle with spring water. "I'll be making tea. Shall I make you a cup, or have you decided to leave?"
For a few moments there was an uncomfortable silence in the room. Then, the tall blonde threw herself into the large armchair with a sigh.- "Two teaspoons of brown sugar, if you've got it. Or I can live with the white stuff if I have to. You're a tough customer, you know that?"
- "Demerara sugar all right? It's a bit fancier than plain brown, but it'll go well with this Earl Grey." At her nod, Woomert added the sugar and carried the cup to her. "As to being tough, I just don't take well to bullying or manipulation. No matter", his eyes swept over her again, "how sweet the bait." Just as she was about to say something, Woomert made eye contact with her again and smiled slightly. "Go ahead - deny it if you will."Auburn snorted and settled back into the armchair, cradling her cup. "OK, so this isn't exactly business attire. I just thought you boys would be a little more... manageable this way." She glanced at Frink, who was still standing by the table in the center of the room. "Come on over and sit down next to me - Frink, is it? I won't bite." Her eyes glittered with amusement. "Not with Mr. Tough here watching, anyway. Or without you asking me first."
Frink walked over and sat down on the far end of the couch. Auburn glanced at him appraisingly for a moment, then spoke. Her voice held no trace of levity, and she was all business.
- "All right, then. Here's the situation: I've had a bunch of hardware failures in a row, which effectively shut down my business. I've got those repaired - in fact, the techs I've hired just got done - but even with the hardware back in place, the software still isn't working, specifically the Apache server. Now, I'm hosting over 500 porn sites on my machines, and they're all losing money by the hour - which means that I am too! What the..." Frink's tea seemed to have gone down the wrong pipe; he choked and coughed, but quickly recovered, waving off all offers of help, and made hand motions that invited Auburn to go on.- "Well, as I was saying - I host a bunch of these things, and now Apache won't come up any more! My techs have tried and tried, but they got nowhere: it seems to be beyond them. They don't know which sites are down without bringing up the server, and they can't bring up the server without... well, you can see the problem. One thing they've mentioned is that there's certainly more than one dead site, at least a dozen - but not much more than that. All I want for now is to get the working ones up, and I'll get the broken ones fixed later - in fact, I know I'll have to restore some of them from tape, since the hardware took a chunk of data with it when it died." Auburn shifted in the chair until she was looking directly at Frink. "What do you think, Big Guy? Can you handle it? The SSH server is still running, so we can still log in."
Frink had finally managed to breathe normally, although his color - due to the coughing, no doubt - was still high.- "Well, Miss Bombshell..."
- "Heck, call me Auburn." She grinned and slid onto the couch next to Frink. "You're kinda cute, y'know. All right, all right," she raised her hands defensively in answer to Woomert's stern look, "I'll behave. But he is!"- "Right. Well... Auburn... let's just log in and see what's going on." Frink walked over to his machine and logged in. After a bit of quick exploration of relevant files, he tried to start the webserver which failed as advertised, and looked through the log file - indeed, there was a list of error messages, but all of them pointed to the first site that was failing. If he were to fix that one, the server still would not come up - the errors would now simply report the next broken site, an iterative process that could take a long, long time. He then looked through the server configuration file, ``/etc/apache/httpd.conf''.
- "Ah. It seems that several of these are misconfigured - this is besides the site structure missing or being broken - but it would take a long time to figure out which is which, and a longer time to fix them all. Auburn, I think our best bet is to do what you suggested - bring up the sites that are OK. However, dealing with each individual entry in ``httpd.conf'' can be pretty painful - so here's what we're going to do..."As he was about to start typing, Frink felt a tap on his shoulder. Looking around he saw Woomert extending... oh, good grief! Those were Woomert's favorite typing gloves! Frink took a deep breath, nodded at Woomert and pulled them on, very conscious of the implied honor.
"First, let's pull these domains out of the file and create references to them; it'll be a lot easier than trying to deal with groups of stanzas. Apache lets us do that with the ``Include'' directive, so..."perl -00wne'/^<V/||next;open F,">",++$a or die"$a: $!\n";print F' httpd.conf
"ls" now showed a list of files numbered from 1 to 509, each of which proved to contain a separate virtual host definition; Auburn gasped and Woomert applauded quietly. Frink grinned to himself; there was more to come.
perl -i -wlpe'if(/^<V/){print"Include $_"for 1..509;exit}' httpd.conf
There was no immediately visible result until Frink opened the "httpd.conf" file in an editor. The "VirtualHosts" section was gone, and in its place was a list of "Include" directives - which would read in the previously-created numbered files.
- "Excellent, Frink!" Woomert was smiling. "What's your next move?"
Frink sat back in his chair, looking flushed with success and a bit jaunty.- "Well, now that I've isolated the vhosts, I get to do a little trouble-shooting - that is, I need to figure out which of these hosts are stopping Apache from coming up. Fortunately, from what Auburn told us, there aren't that many - so it makes sense for me to run a binary split, a.k.a. ``divide and conquer''. First, let's make sure that Apache itself is OK, i.e., that it will come up when all the vhosts are disabled:
perl -i -wpe's/^(?=Include)/# /' httpd.conf
Now, let's try the server... yep, it comes up fine. Let's kill it, enable half the hosts, and try again." Auburn, who by this time had snuggled up behind Frink and was looking over his left shoulder, turned her head and looked at him.
- "Won't that take a long time?"
- "Not at all." Frink's blush was barely noticeable by this time. "The power of the binary technique comes from the progressive splitting, that is, being able to quickly zero in on the problem. For example, if there was only one broken vhost here, it would take no more than nine tests to find it - since 2 to the 9th power is 512, and we have just under that number of hosts. Here, watch..." Frink uncommented the first half of the ``Include'' directives, then tried the server, which failed to start.- "All right - now we know there's a bad vhost somewhere in this bunch. So, we'll comment out half of the vhosts we've just enabled, and... the server comes up, which tells us that the bad vhost is in the group that we've just commented out. Re-enable half of that, and..."
Auburn clapped her hands. "I get it, I get it! Oooh, Frink, you clever, clever fellow!"Frink grinned and blushed again, typing all the while. After a few minutes, he had located fourteen broken vhosts, which he commented out. With a sigh of satisfaction, he started the Apache server for the last time.
- "There. Auburn, it looks like you're back in business. Presumably, your techs can fix whatever is wrong with those broken ones; that's just a question of going through them individually and figuring it out."- "Oh, I'm sure of it - now that you've fixed the main problem. Say, what about somebody stealing my domain names? I tried a few of them while they were down, and came up with some kind of an advertising page! What's going on there?"
- "Oh, I just checked that out; it was only VeriSign practicing trust violation, an odd thing to do given that the only product they sell is reputation-based trust - but stupid greed never looks beyond the short term. They recently decided to do something that was quite unethical, snag any registered but non-resolving names in ``.com'' and redirect them to their own pages." He frowned. "They're under a lot of pressure from the Internet community, so hopefully they'll drop this practice soon. For you, though, there's not much to worry about; once a host is up and answering requests, no redirection will occur."Auburn squealed happily, reached around the chair, hugged Frink, and planted a kiss on his cheek, then jumped up and grabbed her purse.
- "That was terrific, Frink! I'll be sending you boys a check as soon as I get back." She winked. "I've got to hurry, now; I've got a photo shoot coming up for a new site, with me as the star attraction!" She looked at Frink, smiling. "Give me a call sometime, and you just might convince me to give you a private show; I think smart men are sexy. See ya." She skipped over to the door, blew them a kiss, and was gone."Well, partner, I'm proud of you." Woomert relaxed in the armchair and sipped his tea. "You did well, much the approach I would have used - with minor variations, perhaps."
Frink sat on the arm of the couch and flipped over it to the other side, ending up sitting comfortably. He looked somewhat pensive.
- "What would you have done differently, Woomert?"- "Oh, nothing significant; a matter of style. Five hundred-some vhosts isn't a terribly large number - I think I would have run a loop that added vhost ``includes'' one at a time to ``httpd.conf'', restarted the server, and commented the last entry if it failed. This way, the identification would be completely automatic, and it wouldn't matter if the number of bad vhosts was high relative to the total - a scenario where a binary search would become more time-intensive than a linear one. However, your approach works better where the total is large and the number of bad spots relatively small, which was the case here.
I'll tell you what. I know that you came up with this solution all on your own, and it's a good one... but why don't you go ahead and explain it to me? Not only am I interested in how you code - that's obviously critical in what we do - but in how you think of the problems and the approaches to them. Pretend that I don't know anything about it, and feel free to go in-depth on anything you feel to be relevant."Frink straightened up, smiling.
- "Woomert, sometimes I think that you're reading my mind. I was just thinking how I'd miss these ``code dissection'' sessions, where I had a chance to ask you about any ambiguities or fine points; this takes care of that, and anything I'm not sure of doesn't have to be handled in front of the clients.All right, then, here we go. We'll start with my first one-liner:
perl -00wne'/^<V/||next;open F,">",++$a or die"$a: $!\n";print F' httpd.conf
Obviously, I enabled warnings with ``-w'', started a non-printing loop which read the specified file with ``-n'', and executed the code with ``-e''; however, the ``-00'' takes a bit of an explanation. You see, Perl allows you to specify an ``input record separator'', or what it considers to be an end-of-line character. By default, it's ``\n'' - but we can redefine it, either by changing the value of ``$/'' within the script, or specifying a ``-0x'' option where ``x'' is a hexadecimal or octal value specifying the new EOL character. Furthermore, ``-00'' is special: it will cause Perl to read the file one paragraph at a time - a paragraph being defined as a block of non-blank lines followed by one or more blank lines. Since the format of the ``vhosts'' portion of ``httpd.conf'' is laid out like this:
<VirtualHost ip.address.of.host.somedomain.com>
ServerAdmin webmaster@host.somedomain.com
DocumentRoot /www/docs/host.somedomain.com
ServerName host.somedomain.com
ErrorLog logs/host.somedomain.com-error.log
CustomLog logs/host.somedomain.com-access.log common
</VirtualHost>
<VirtualHost ip.address.of.host.somedomain.com>
ServerAdmin webmaster@host.somedomain.com
DocumentRoot /www/docs/host.somedomain.com
ServerName host.somedomain.com
ErrorLog logs/host.somedomain.com-error.log
CustomLog logs/host.somedomain.com-access.log common
</VirtualHost>
...
I knew that ``slurping by paragraphs'' was the right thing to do." - "Well done! What about the script itself, then?"
- "Well, I wanted to skip everything in there until I did get to the "vhosts" section. Having glanced through Auburn's ``httpd.conf'', I knew that nothing in there began with ``<V'' until you got to ``vhosts'' - so I just told Perl to skip everything until then; ``/^<V/||next'' says ``unless the current line begins with a "<V", jump to the next line.'' Once it got to the right section, it would actually become a double-check: all the paragraphs we wanted were supposed to begin with a ``<V''.
Going from that point forward, I started creating files by using Perl's three-argument version of ``open''. You see, I wanted to increment the number which would be the name of the file, but I also needed to specify the ">" prefix - and I knew that I couldn't just say ">$a++", since the increment operator wouldn't work when quoted. However, the three-argument version of ``open'' has been around since Perl 5.6, and it came in very handy: ``open F,">",++$a'' worked fine. Of course, I tested the return value - just as you've taught me to do with any significant system command - with ``or die'', and specified that it should print the system error ($!) as well. Last of all, having created the ``F'' filehandle, I printed to it - and, as always, when ``print'' does not receive an argument (except for a filehandle), it prints ``$_'' by default. Given the ``-n'' and the ``-00'', this would contain the current paragraph, that is, the entire ``VirtualHost'' stanza. How's that for an explanation, Woomert?"Woomert laughed happily. "I'm proud of you, Frink. You've come a long way, just as I'd said before. Go ahead and run over the rest of what you've done; I have a wonderful lunch almost ready for us, but we've still got a few minutes."
Frink leaned back on the couch."Well, the rest is even easier - although admittedly a bit more dangerous.
perl -i -wlpe'if(/^<V/){print"Include $_"for 1..509;exit}' httpd.conf
Knowing that I'd be doing this, I had actually made a backup copy of ``httpd.conf'' before even the first operation, something else I learned from you. Accidentally losing that would have been embarrassing as well as big trouble! However, once that was done, I felt safe in using the ``-i'' switch - it does in-file editing. Where the output of the Perl script would normally go to STDOUT, it would now be written to the file being read from; this is very handy but dangerous, as I've found in the past when I used ``-n'' (the non-printing loop) and forgot to specify a ``print'' statement in the script! Anyway, I double-checked that everything was OK this time, and - once again - went looking for a ``<V'' at the beginning of the line. As soon as I found one, though, I just printed what I wanted and threw away the rest of it: ``print"Include $_"for 1..509'' would loop over all those numbers and print them with ``Include '' as a prefix, with the ``-l'' switch appending an end-of-line character to each ``print'' statement. As soon as that was done, ``exit'' dropped us out of the loop - and voila, an updated conffile!" Woomert walked over to the oven and peeked in through the glass window. "Ah, almost done. Go ahead, Frink - I'm listening very carefully."
- "Well, the last one is pretty simple - although it uses a feature of the regular expression set that beginners might not know about: one of the so-called "extended" patterns, a ``positive look-ahead assertion''.
perl -i -wpe's/^(?=Include)/# /' httpd.conf
The ``^(?=Include)'' pattern matches the beginning of any line which is followed by ``Include'' - but does not, erm, include the ``Include'' in the string to be replaced. That is, the substitution expression ``s/^Include/# /'' would have replaced every ``Include'' at the beginning of a line with a hash mark an a space, whereas ``s/^(?=Include)/# /'' simply puts those characters at the beginning of the line. ``s/^Include/# Include/'' would have done the same thing, but I didn't think it was very elegant."
- "Very well said, Frink! Elegance in code is important - even in one-liners, which - shall we admit it? - are quite ugly except for their functional appeal. Well, we do what we can. You hadled it very well indeed.
Now, for lunch, we've got a bit of Dutch-Indonesian cuisine: I'm setting out a rijstaffel - ``rice table'' to the uninitiated, the idea being something like a Swedish smorgasbord, a pick-and-choose sort of thing. There are eight different kinds of satay (tasty skewers of various meats), ajam panggang (a sort of barbecued chicken) with nasi goreng (fried rice), gado-gado salad, and of course lots of peanut sauce, ketjap manis, and chili paste if you want a bit more spice, always a doubtful proposition with, erm, properly-seasoned Indonesian food. There's also Thai iced coffee to follow - not Dutch or Indonesian, but I find it does well for putting out the fire!"
"Oh, and speaking of fire... you seem to have made a friend today." Woomert grinned as Frink blushed and started to stutter his way through some complicated explanation that began with "Why, I would never...", and held up a warning finger. "Shush. You will end up doing whatever you decide is right for you, hopefully after careful consideration of the issues involved - and I will offer you my advice if you wish it, but not right now. For the moment, we have this excellent food waiting for us - and the only advice I will give you is to watch your elbows. Some of those sauces will eat through the table and possibly the marble floor in no time at all..."
Ben was born in Moscow, Russia in 1962. He became interested in electricity
at the tender age of six, promptly demonstrated it by sticking a fork into
a socket and starting a fire, and has been falling down technological
mineshafts ever since. He has been working with computers since the Elder
Days, when they had to be built by soldering parts onto printed circuit
boards and programs had to fit into 4k of memory. He would gladly pay good
money to any psychologist who can cure him of the recurrent nightmares.
His subsequent experiences include creating software in nearly a dozen
languages, network and database maintenance during the approach of a
hurricane, and writing articles for publications ranging from sailing
magazines to technological journals. After a seven-year Atlantic/Caribbean
cruise under sail and passages up and down the East coast of the US, he is
currently anchored in St. Augustine, Florida. He works as a technical
instructor for Sun Microsystems and a private Open Source consultant/Web
developer. His current set of hobbies includes flying, yoga, martial arts,
motorcycles, writing, and Roman history; his Palm Pilot is crammed full of
alarms, many of which contain exclamation points.
He has been working with Linux since 1997, and credits it with his complete
loss of interest in waging nuclear warfare on parts of the Pacific Northwest.
Ben is the Editor-in-Chief for Linux Gazette and a member of The Answer Gang.
