I just can’t leave well enough alone. I’ve been mildly annoyed with the “hash in progress” and progress dialogs in WinHasher 1.1. The original idea was to use System.ComponentModel.BackgroundWorker to easily multi-thread very large hashes (say of CD or DVD ISOs or uncompressed video files). This had two benefits: (1) it allows the user to cancel a hash in progress and (2) gives us an opportunity to update the GUI while the hashing takes place in the background, meaning we can inform the user of the progress. Unfortunately, I couldn’t find a method right away to determine the progress of an individual hash. System.Security.Cryptography.HashAlgorithm.ComputeHash() by default takes a byte array or file stream and chugs the whole thing at once, spitting out the hash as a result. There’s no way with this method to determine how far along you are.
However, if you look at the guts of ComputeHash(), you’ll find it reads in chunks of bytes into a buffer, then calls two methods: TransformBlock() for every chunk but the last, and TransformFinalBlock() to hash the last chunk and finalize the hash. The result can then be obtained from the HashAlgorithm.Hash property. If we bypass the convenience of the single ComputeHash() method call, you can read chunks of bytes from the buffer, feed it to the Transform...() methods, and keep track of how many bytes have been read so far. Since we already know how big the file is from the start (System.IO.FileStream.Length), it’s trivial to calculate a percentage complete. Want the progress of a multi-file comparison? Sum the lengths of all files in the batch, then keep track of the total number of bytes hashed along the way.
I’ve bumped WinHasher to version 1.2. It should be available on the official site by tomorrow morning.
I don’t usually do link-and-run posts (I prefer to have actual content in a blog), but I thought this was disturbing enough to disseminate. I’ll probably add my own blathering commentary which will make it more than a link-and-run post anyway. (After all, I know all of you who come here really come for the blathering. I’m just so blatherful….)
I’m not sure how many of you out there follow the Security Now! podcast over at TWiT, but it’s probably obvious by now that I do, given recent posts. This past week’s episode, #119, exposes a rather unsettling fact that shouldn’t be ignored. (The high quality 64kbps MP3 can be found at that link, while a 16kbps MP3, a transcript in various formats, and additional notes can be found here.) While I encourage you to download and listen/read the facts for yourself, I’ll see if I can summarize it below for the attention-span impaired.
For a long time, I’ve defended PayPal as a method of monetary transfer. They’ve always been good to me personally, even during the stormy periods where some GPF readers boycotted them for “questionable” practices. (See the PayPal Wikipedia entry for an abbreviated history.) For that matter, many online comics wouldn’t be able to monetize themselves in any fashion if it weren’t for PayPal, as many webcomics use the service for donations and online stores. (PayPal has always been an acceptable form of payment in every incarnation of the GPF Store.) They’ve always had issues with customer service, but they’ve also been champions in anti-phishing campaigns.
But Steve Gibson and Leo Laporte have helped disclose a rather shady new practice: In a previous Security Now! episode, a listener mentioned problems downloading a software service from PayPal, only to discover that the download link was sending him to a server over at DoubleClick rather than PayPal. Since he was locally blocking access to the domain “doubleclick.net” in his hosts file, the link failed and the software would not download. Gibson promised to investigate the incident and after a number of side-tracks finally presented his results.
DoubleClick, for the few out there unfamiliar with it, is one of the Internet’s largest online advertising agencies, serving ad banners to millions of Web sites (including, indirectly, GPF). DoubleClick has long been unpopular among netizens for its questionable policies of tracking Web surfers across multiple sites, using a trick with tracking cookies to follow you from site to site. Privacy concerns were raised even further when Google, a company that itself stores and indexes a lot of personal information about its users of GMail, Ad-Sense, and other services, recently purchased DoubleClick. DoubleClick eventually bowed to pressure from the Net at large and created an opt-out page so their tracking cookie would contain “non-personally-identifiable information” and thus negate some of the tracking cookie’s effectiveness. (This opt-out page is still linked to (now indirectly, as the URL has changed) from the GPF privacy policy page.) Many folks these days, however, including myself, simply run spyware scanners like Spybot: Search & Destroy or Ad-Aware and periodically delete such tracking cookies, or just block the “doubleclick.net” domain and its subdomains using the hosts file trick mentioned above. (This is how, in part, Spybot’s immunization against cookies works.) This eliminates or at least minimizes the opportunity for your Web surfing habits to be linked personally to you.
However, PayPal’s new links bypass many of these anti-drive-by-cookie-ing techniques by sending you directly to DoubleClick’s servers, rather than inlining content like Flash or images from their site. Since these are internal PayPal URLs and not links that are expected to send you to the outside, they should be immediately suspicious. What’s even worse is that if you examine the URL closely, there appears to be some sort of “user ID” like number included that may personally identify you if you click on it. What’s even more disturbing is the number of these links you run across as you surf the PayPal site; while some obviously ad-like images contain the “doubleclick.net” URL, many links in the site bar that look like ordinary navigational links contain it as well. While Gibson points out–quite rightly–that there is no evidence to support any sort of conspiracy theories that many come to mind, it is obvious enough that some sort of information sharing is going on between the two companies, and that if a unique user identifier is indeed being passed along with the URL, there’s a likelihood that both companies can link your potential spending habits with PayPal to your surfing habits tracked by DoubleClick.
Now it’s easy to be alarmist and to say everyone should boycott PayPal. Unfortunately, so many of us in webcomics depend on PayPal for survival, so there’s no way we can easily remove ourselves from it. And there’s no competitor out there with enough critical mass to really challenge PayPal for dominance, so there aren’t many viable alternatives. Thus the only current immunization option is diligent observation.
The good news is that the DoubleClick URLs within PayPal’s site all contain at the end PayPal URL you will eventually be redirected to. It’s trivial to copy the URL, paste it into your address bar, crop out the DoubleClick portion, and go directly the the PayPal internal destination. Laporte even suggested that it won’t be long before someone comes up with a Firefox plugin that does that for you on the fly. The problem I see with this is that it won’t be long before the diabolical duo figures out savvy users are bypassing the links and they find a better way to obscure the redirection target URL so the copy/paste/edit trick will no longer work. While true encryption might be a bit too much server load for them to handle en masse, a simple ROT13 or Base64 encode might be enough to thwart all but the most stalwart gearheads.
So… should you avoid PayPal? That’s up to you. I can’t, but I’ll be a lot more careful of where I click on their site from now on.
Okay, so the subject line kinda screams “I’m a n00b!” But at least it’s true.
I had a new task added to my plate recently. We’re in the process of consolidating some servers at work, moving some of the work to virtual machines using VMware. Like just about everyone one else in the tech industry who hasn’t been living under the PDP-11s, I’ve heard all the hype about virtualization and how it’s going to revolutionize everything there is to revolutionize. Naturally, the idea sounds cool. Who wouldn’t want to create a “virtual computer” living inside their current hardware that you could simply delete and recreate as easily (almost) as if you were working with a simple text file? Of course, all I’ve really been exposed to until recently was the hype, and maybe what little I know about virtual machines and emulation that I’ve picked up from playing with Java, MAME, and DOSBox.
Well, I was recently assigned the task of creating a new virtual Linux box (openSUSE for the nosy distro snobs out there) to serve as our new source control server, using primarily Subversion and Trac. (Yes, we’re a Microsoft shop using Open Source source control and tracking tools. I enjoy the irony too.) It was a bit surreal firing up VMware and creating this new, clean slate to install upon. You specify what type of virtual processor to use (and how many), how much memory and disk space to allocate, and click the start button. Hello, BIOS! The setup took a bit longer that I expected (mostly because we downloaded packages from an online repository instead of just downloading the DVD ISO) but within a few hours I had a brand new openSUSE “box” ready to go. I’ve been configuring it off and on for several days now, setting up SVN and Trac and finding the best way to migrate our existing projects over without losing any ticket or module ownership data. But the wildest part of it all is that once I’m done building this thing, I can just export the virtual machine files to the new, final host and kick it off again, and it’s as if the machine had always existed there. Surreal, I repeat. Surreal.
About the only thing keeping me from creating a blue billion virtual computers around the house is (a) time and (b) system resources. (I doubt I can justify leaving that many virtual machines running at once.) Geek fun!
Follow up from the last post: Comments should be working again, although not in a fashion that I’d prefer. I had to completely disable Admin-SSL, thus login and login cookies are no longer secure. If you logged in while it was active, you’ll likely need to log in again before you can post, even if you check the “remember me” box. I’m still looking into why it doesn’t work. There’s a known limitation that shared SSL setups will fail to recognize logins in insecure mode, but they say nothing about dedicated setups like mine, which I would think would imply that it ought to work. Then again, I also had some mod_rewrite rules in place, but I experimented with removing those and so I don’t think they’re part of the problem.
I suppose this isn’t a big deal for you guys, since I doubt anyone will be all torn up over making sure their logins to some random guy’s blog are secure. It affects me more than anyone else, because I’d feel much safer if I could protect the admin parts of the site with a layer of encryption. Still, in places where I really need encryption I can verify it still works, so I guess the blog will just have to be open. :\
It has come to my attention that comments don’t appear to be working here at the moment. I strongly suspect this is due to the Admin-SSL WordPress plugin that I’m currently using to secure parts of the blog. It seems to forget that you’re logged in when you’re not accessing a secure URL (i.e. you’re not in SSL mode). I’ve tried a few of the workarounds suggested, but none are working. I have a hunch on a few things to try, but that will have to wait until I have more dedicated time to work on it. (I’m currently watching the little guy by myself at the moment.) I’ll post an update when I think it’s been fixed.
The above picture was sent to me by GPF readers Jeremy and Teresa the other day. Accompanying the pic, Jeremy wrote:
BTW, I don’t know if I sent this to you last summer, but I meant to. My wife and I had a Nick & Ki figure made for our wedding “cake topper” in July, 2006. We had Nick’s hair done blonde to match mine, and I thought it looked great.
And it does. Another fine example of Lanin Thomasma’s excellent handiwork. If you haven’t seen his awesome GPF sculptures, then you need to go give them a look. Lanin has crafted a good dozen (or more) GPF and non-GPF maquettes for me personally, some of which have accompanied me to shows & cons. I don’t really want to turn this post into an advertisement, but these sculptures are so incredible it’s hard not to.
As for the statue in question, Nick looks a little odd to me as a blond, but it definitely fits with the rest of their wedding photos. (Jeremy was gracious enough to share them with me.) I hope you two have an awesome future together. I certainly hope nobody at your wedding was kidnapped by any “evil twins” from alternate universes….
(Random trivia: The “cake topper” for our wedding was Mickey & Minnie Mouse, since we were going to Disney World for our honeymoon. While Nick & Ki existed at that point and I knew they were eventually destined to be married, nobody outside of a circle of close friends and family had read the GPF scripts yet and I certainly didn’t know of anyone that could make custom maquettes for us. Oh, well….)
I was downloading Fedora 8 yesterday with every intention of (eventually) upgrading Demeter, our Linux box, currently running Fedora Core 5. I say “eventually” because finding time to back everything up, install the new OS, restore all the files and third-party apps, and then spend hours debugging what went wrong is the hardest part of the whole process.
Anyhoo, I thought I’d put good ol’ WinHasher to use to verify the SHA-1 hash and make sure the download was successful. That was, after all, the primary reason I wrote the thing. But then I noticed something incredibly annoying. The DVD ISO for Fedora 8 weighs in at around 3.18GB. WinHasher has no problems hashing such a file as it hashes the file stream as it reads it, so it can hash any file of arbitrary size. But while it’s doing so, there’s absolutely no feedback to the user. In fact, if you use the right-click “Send To” shortcut, unless you go into the Windows Task Manager and check the process list, you won’t even know the thing is running. It seems to go off into la-la land until mysteriously, a minute or two later, it comes back with a mysterious pop-up box with a cryptographic hash in it.
Bad programmer! Bad!
Well, this annoyed me enough to fix it, so I’ve tweaked it now so it actually runs multi-threaded. This is ridiculously easy to do when you use the System.ComponentModel.BackgroundWorker object in .NET. You just create the object, create a few event handlers for it, and tell it to go do its thing. The hardest part is if you want it to report its progress; then you have to tweak your working code to generate a percentage complete so it has something to display. I learned this trick when I was working on Mandelbrot Madness! 2.0; I have no idea why it never occurred to me to incorporate it here. The annoying part about this addition, though, is that there’s no way to report on the progress of an individual hash. System.Security.Cryptography.HashAlgorithm doesn’t provide a way for you to know how far it’s gone, so there’s nothing to feed to BackgroundWorker.ReportProgress(). The comparison feature does, however, let you know how many of the files have been processed so far in the batch.
So if you’re one of the few brave souls who downloaded WinHasher 1.0, please grab 1.1 at the official site. It officially goes up tomorrow (a delay caused by Keenspot’s parsing to insert the advertising codes). If you haven’t already downloaded it, why not give it a try?
I mentioned last week that I was working on a neat Apache mod_rewrite trick for locking down access to certain administration pages, but that I wasn’t having much success with it. Well, it seems to be working now and, as promised, I wanted to share it with anyone who might be interested. Fair warning to non-technical readers: extreme geekery lies ahead.
First and foremost, I can’t claim full credit for this idea. It borrows some from Steve Gibson’s roaming authentication scheme outlined in episode #113 of the Security Now! podcast. In that show (and subsequently continued in episode #115), Gibson outlines his method of allowing his employees to access secure portions of his site while traveling. The method described here is not quite as secure as his, as I’m forcing things to happen at the Web server software layer as opposed to the application layer and thus don’t have the same fine granularity of control he has. However, it uses many of the same ideas.
It’s relatively easy with mod_rewrite to protect certain resources of a site by restricting access to certain IP addresses. Consider the following:
RewriteCond %{REQUEST_URI} ^/store/admin/.*
RewriteCond %{REMOTE_ADDR} !^192\.168\.13\.
RewriteCond %{REMOTE_ADDR} !^127\.0\.0\.1$
RewriteRule ^/store/admin/.* /store/ [R,L]
This rule set essentially says: (1) if the requested URL starts with the string “/store/admin/” and (2) the IP address of the requesting client does not begins with “192.168.13.” or (3) is not exactly “127.0.0.1″ then (4) redirect all requests for URLs starting with “/store/admin/” to the root URL of the store, “/store/”. Essentially, we’re only allowing access to what is apparently the administrative portions of an online store to a very limited number of IP addresses, one of which is fully qualified (the “loop-back” address of 127.0.0.1) and the rest belonging to a range (192.168.13.0 through 192.168.13.255). Anyone outside these IPs will be transparently redirected to the front page of the store. (Redirecting is much friendlier than outright forbidding access.) All of this takes place in Apache itself, before we even get to the application and any potential security flaws it might have. There are no worries about hacking the store software itself to deny access. Of course, we can list any number of REMOTE_ADDR entries that we wish; each condition is a regular expression (which are negated here by the “bang” at the front) so we can filter on any octet we want and can easily specify real, outside IPs rather than private ones. For example, for this site I limit access to my various admin sections to the IP of my cable modem and our outside IP at work.
However, what happens when you are required to go on a trip and need to access the administrative parts of the site while on the go? Obviously, you can’t add the hotel’s outside IP to this rule set in advance (imagine asking the front desk for that information), and you probably won’t be able to add it easily once you get there. Sure, WordPress and the store front software have login security on their various admin interfaces, but we’re trying to protect those from hackers, right? Aside from reopening them to the entire Internet before the trip and closing them again once we get back, there aren’t very many options. How then can we identify approved “roaming” users and/or machines so they can access the admin sites without being inside a hard-coded list of IPs?
Gibson’s answer was to optionally set a secure cookie in the user’s browser if they access the admin site within one of the approved IPs first. Being within an approved IP, they aren’t restricted by the access rule and they are allowed to reach the login prompt. During login, they are prompted on whether or not they want to enable roaming access on this particular machine. If they agree, a secure cookie is set in the browser and set to expire at some date in the future. Later, when the user attempts to access the admin site outside of the approved IP list, the site checks to see if the cookie has been set. If present, the user is allowed to log in, just as if they were within one of the approved IPs. The cookie acts as a kind of two-factor authentication: the first factor being “something you know”, the user name and password, and the second being “something you have”, the cookie. Since the cookie is set in secure mode (HTTPS), it will only be sent back to the site over a secure connection. And since (well behaved) browsers only allow a site to read the cookies it has itself set, no other site should be able to read it.
This is all well and good… if you have access to the source of the application you’re trying to secure and you’re willing to hack it. Gibson wrote his own store front, so this was relatively easy for him to integrate. But I want to secure WordPress, a third-party store app, and a few random subdirectories that are pretty much statically built HTML. As much as I like running Open Source software, I usually prefer not to muck around with things if I can help it, lest I screw something up. Thus, I don’t particularly want to hack WP and the store to add this extra layer of functionality. Fortunately, though, mod_rewrite gives us a mechanism through which we can accomplish basically the same thing without modifying the underlying application. In theory, since all this occurs before we even reach the application, one could argue it may even be more secure than the application’s authentication mechanisms themselves.
You can actually set browser cookies via mod_rewrite rules. Consider what happens if we insert the following before the rules we defined above:
RewriteCond %{REMOTE_ADDR} ^192\.168\.13\.
RewriteCond %{HTTP_COOKIE} "!(.+; )*admincookie=uniqueval(; .+)*"
RewriteRule .* - [CO=admincookie:uniqueval:.domainname.tld:43200:/store/]
This rule set essentially says: (1) if the remote IP starts with “192.168.13.” and (2) there isn’t a cookie already set by the name “admincookie” then (3) set a cookie named “admincookie” with the value “uniqueval” for the domain “.domainname.tld” (assuming that’s our real domain name) for a period of 30 days (60 minutes x 24 hours x 30 days = 43,200 minutes) restricted to the path “/store/” and its subdirectories. Now let’s modify the rule set from before:
RewriteCond %{REQUEST_URI} ^/store/admin/.*
RewriteCond %{REMOTE_ADDR} !^192\.168\.13\.
RewriteCond %{REMOTE_ADDR} !^127\.0\.0\.1$
RewriteCond %{HTTP_COOKIE} "!(.+; )*admincookie=uniqueval(; .+)*"
RewriteRule ^/store/admin/.* /store/ [R,L]
Note that we’ve added a new condition. In addition to checking for the approved IP list, we also check to see if the “admincookie” has been set and that its value is what we expect (”uniqueval”). Note the parenthetical parts at the beginning and end of the cookie regex; these should make sure we match the unique cookie name/value pair, regardless of how many cookies are present. (Also note the quotes around this regex; since whitespace delimits the parts of the rewrite statements, the quotes are required to include the spaces after the semicolons in the regex. Without the quotes, the regex produces a “bad flag delimiters” error when Apache parses the configuration file.) Since each approved item’s entry is negated, the rule is only applied if none of them match. So now we should be able to get into the site remotely if and only if we’re inside an approved IP or we have the secret cookie, which we know is only set if we’ve been in one of the approved IPs first. Instant roaming authentication!
To summarize, the primary advantages to this scheme are:
There are, of course, a few caveats:
mod_rewrite to force certain URLs to always use SSL (assuming you have a secure certificate), thereby securing the connection first. All WP admin functions, the GPF Store, and my other secured admin locales here on this site are all secured via SSL, so that helps in keeping my site secure by eliminating sniffing. (Of course, if you go this route, don’t forget to copy any necessary rules from the main Apache configuration file to the SSL config file, as the secure site will be treated as a different virtual host with its own set of rewriting rules. This little hiccup is what was keeping me from publishing this for quite a while.)mod_rewrite does not have the facility to specify secure mode in a cookie set by a rewrite rule. Thus, the above cookie is not secure and will be sent with each request in or below the specified path, encrypted or not. The cookie is then theoretically susceptible to sniffing attacks. Setting a secure mode cookie is easy enough to do in application code, but not apparently so in mod_rewrite.mod_rewrite. (Remember, all this is occurring in Apache before we even reach application code.) Right now, %{HTTP_COOKIE} variable gets all the cookies for a given site/path as one big string, with each name/value pair delimited by a semi-colon and a space (”; “) and the name and value are glued together with an equal sign. I’m looking into a better regex to match this more precisely and I’ll update this post if I find one.I welcome any feedback on how to improve this, especially if anyone knows how to get around the secure and unique cookie caveats.
Appendium: I should also point out that this scheme should be equally usable if you place the code in your master Apache configuration file (usually something like /etc/httpd/conf/httpd.conf on UNIX clones) or in per-directory .htaccess files. I usually prefer to put such rules in the master config file, mostly because it’s more secure (outside of the document root) and only gets parsed and loaded once while .htaccess files are read and parsed each time there’s a request in that directory (or any of its subdirectories). However, that only works if you have access to the master config, which most shared hosting services don’t provide. Of course, such rules placed in an .htaccess file will only apply to that directory and its subdirectories, so you’d have to tweak the rules (such as file paths and the cookie path) as necessary.
Update 11/20/2007: Updated cookie regex to better match the exactly name/value pair; added notes about rotating cookie values.
Update 11/30/2007: Put cookie regex in quotes to correct avoid “bad flag delimiters” parsing errors; added advantage summary to better showcase the advantages of the scheme; updated my cookie value scheme; added highly-random subdirectory alias to avoid unintentional cookie-ing
You know you’ve officially earned your fatherhood stripes if you can say the above sentence with a straight face and a casual, nonchalant inflection.
One of the first real trials by fire any parent has to face is the dreaded poopy diaper. I’ve been told that babies have no concept of things smelling “good” or “bad”; they only differentiate things as smelling differently. This is probably good since they could spend an unspecified amount of time sitting in their own filth until their guardian finally finds out what happened and takes care of it. Nevertheless, we as a species eventually developed a total aversion to the odor of our own feces, most likely as a mechanism to make sure we don’t accidentally consume any in our quest for sustenance. As a general rule, things that smell bad aren’t very good to eat, certain types of fragrant cheeses not withstanding. Many of us in today’s modern, high-tech, hyper-sanitary world tend to forget this fact, as we rarely need to use this knowledge for survival. We forget it, that is, until it comes flying right back into our faces–in some cases, literally–when we start reproducing.
Anecdote #1: My wife called me in yesterday morning to help change the boy’s diaper. He had just awakened from a nice, restful, 12.5 hour night’s sleep (lucky him!) bearing a full load (OK, not so lucky). Having obtained such a high quality and quantity of sleep, he was in no mood to lie there peacefully. He was in full go-mode with the wiggle quotient cranked to high, babbling happily with every other word being “Da da”. In such situations, one of us takes the low road and performs the actual toxic clean up while the other takes the high road and does their best to restrain the flailing legs as he kicks and squirms.
Well, my wife extracted him from the crib and was on clean-up duty when she ascertained that she required my assistance with restraint. So I walked into the room and instinctively reached out and grabbed the pair of waving ankles before she even had the chance to start uttering her warning. “He just stuck his ankle in it,” she said sheepishly, still wiping frantically at his bottom. I extended my index finger into view. Yep, there it was, right there on the tip of my finger.
I’m not sure if it was the fact that I hadn’t had my morning shot of caffeine yet or just the jaded numbness of parental experience, but all I could manage was, “It’s all right, it’s just poopy.” My wife nearly snorted with laughter. I very calmly remained at my station until the clean-up was completed, the diaper was closed, and the boy secure. I then proceeded calmly yet briskly across the hall to the bathroom and scrubbed until my finger was raw.
I have a bad habit of biting my fingernails. I think I kicked that habit, at least for the day.
Anecdote #2: Hours later, we’re about to head out for some longer over-due shopping. It’s usually my job to carry the boy downstairs to the car and buckle him in while my wife gathers all his various accessories (diaper bag, toys, etc.). Right now the two burdens are about equal in weight, but he’s steadily increasing in mass, giving me the weightier load. I hoisted him up to my hip in the typical carrying configuration, and a faint yet unfortunately familiar scent wafted to my nostrils.
My wife undoubtedly noticed the odd look on my face. “What’s wrong?”
“I smell poopy.”
“I just changed him.”
“Well, it’s either the boy or Daddy, but I definitely smell it.” I spun him around, lifted him higher, and planted my nose into his tush. (This maneuver is not recommended; only trained professional parents should attempt this.)
“Powder fresh.” I didn’t like the implications of this deduction.
Woo.
You know, I could wax all philosophical, get nostalgic and misty-eyed, ramble on and on about how things have changed between 1998 and now, but… I’d rather not. If you care, celebrate in whatever means you are so accustomed, although I request that if you do so in the name of GPF that you do so in a socially responsible manner. (Don’t want anyone going to jail for some drunken GPF-inspired murder spree.)
I guess I could mention, in the light of other comics celebrating a decade of existence recently, that GPF was actually created in 1997 and I spent over a year developing and honing it before it came online. So in a sense, GPF is also a decade old, although not officially until November 2, 2008. Somehow, I doubt anyone would really count that.
Woo.