It would be nice if OS X machines that downloaded patches via Software Update could make those patches available to other OS X machines on the same network. Especially since some of the patches nowadays are very large (the latest Leopard patch was a whopping 180MB) and it takes time and bandwidth away to do it.
Larger sites are served by running an OS X Server somewhere on the LAN, and running Software Update Server on it. Then you can either fake DNS entries or change client settings to have all your clients use the centrally downloaded patches.
But what about a small, client-only Mac network that a lot of people have at home? e.g. an Mac Mini and a couple of MacBooks. Perhaps there is a way for Macs to play nicely together if you've got things like Bonjour turned on (I don't), but alas, I've never come across it. Please let me know if there is a way.
In the meantime, I created a centralised patch directory on my LAN file server to save all the dowloaded patches to once one machine had done the hard work (using the "Install and Keep Package" option of Software Update). When selecting this option, the package is saved to /Library/Packages on Tiger and /Library/Updates on Leopard. I then copy the files over the network (I use rsync, but you could just as easily share the directory and mount it via AFP, SMB, NFS, etc), ready for download on the next Mac. If you were really adventurous, you could even try NFS mounting the directory as /Library/Packages directly on each machine... let me know if anyone does this and it works ok. :)
A further note here; I set all my Macs to not automatically download patches in System Preferences, and on all but the "primary" Mac I disable the automatic checking to see if patches are available. The idea is that I want to know patches are available on the primary Mac, and I will then "Install and Keep Package" for each patch and make sure it is saved to the network before going on to the next one. This I learnt the hard way, in that on my Leopard MacBook it seems to want to clear out /Library/Updates between reboots for some reason... I haven't had time to look into why.
I know what packages need to be downloaded on to the second and subsequent Macs by making sure the /Library/Packages directory is cleared on the primary Mac once all the patching is done. Then when new patches are available, I just get the directory listing from /Library/Patches to know which ones to copy over to every other Mac (then delete and clear this directory again, etc). You need to do this as some patches (e.g. iTunes) are just saved as a package like "iTunesX.pkg" in the directory. Each new version of iTunes gets saved over the top of this package. The last modified time of this top level directory is usually unchanged, meaning that you can't tell a new iTunes patch is available just by looking at the date stamp.
It's also a shame that Software Update doesn't seem to look at /Library/Packages automatically for you to see if you've already downloaded the patch. It keeps it's own (per-user) cache files for downloads and ignores this central directory for some reason, even though it gives you the option of keeping the patch here after installation. Not very user-friendly, is it?
Let me know if you've got a better way of keeping your small home network of Macs patched without repeatedly downloading patches. Oh, and before you suggest just caching the patches in a small home network proxy server (also suggested in the macosxhints.com thread above), I already do. :) However, this relies on changing settings on your proxy server to accommodate some rather large files that you mightn't otherwise want to do. And it isn't fool-proof... for some reason (probably related to the delay in between patching machines) I've had Macs on my network download the same patch twice, even though the first download should have been cached. Besides, the whole "play with Squid" route to do this starts to just get too complicated, especially when it seems like it should be very easily handled via file sharing or support within OS X itself....
Monday, 25 February 2008
Tuesday, 19 February 2008
Qt and MySQL plugin issues
Lately I've been playing with a recipe management application called krecipes. I had used the FreeBSD port in the past, but I really wanted to run it locally on various Macs that I have and have them all connect back to the central recipe database on the FreeBSD server.
And so I needed to venture into the mess that is the current Mac open-source ports/packaging system. I say "system" (singular) deliberately; I well know there are many options available in this space, but there really shouldn't be. Good operating systems tend to really only have one system that everyone uses to install and manage open-source software so that effort is not duplicated unnecessarily (which is exponential due to the thousands of applications that need to be tracked in any packaging system). Any gripes with how the current system works should be sorted out within that system and let it evolve over time rather than splitting off yet another effort to do ports/packages "a better way".
Apart from the aforementioned waste of effort by port maintainers, it's just down right confusing for users (even experienced ones). The Mac is a perfect example. It has one of the best application management systems from a end-user point of view (drag and drop to install and delete - how easy is that!) and any system that provides access to open-source software should try and be that easy as well. But if you want to tread in this space you are met with a bewildering array of options; Fink, DarwinPorts, MacPorts, RPM on MacOS X, and it wouldn't surprise me if there are others as well.
I think there may be some link between DarwinPorts and MacPorts, but quite frankly, that proves my point about how confusing it is. These sites exist, various people advise using them for different reasons, and there really isn't any definitive advice on which one you should use. The web sites for these systems are universally bad as well - there just isn't clean information on any of them about why you should use their system and what the key differences are between the systems. Unless you want to spend hours and hours researching the history of these systems you're really left in the dark. And quite frankly the whole point of these systems is to save you hours and hours and not force you to waste hours and hours.
Ok, rant finished. :) For some reason in the past I had actually installed Fink on my test Mac, so I started there. As it turns out, this is the only system which has a port for krecipes anyway... *sigh* Unfortunately, it's been setup to compile and link against an older version of Qt (3.3) that compiles as multi-threaded (good), but the sub-port for the MySQL plugin compiles as single-threaded (bad). So krecipes doesn't work and spits out the following error message as expected:
Conflict in /sw/lib/qt3/plugins/sqldrivers/libqsqlmysql.so:
Plugin uses single threaded Qt library!
After no response from the port maintainer in 1-2 months (what a surprise) I decided if I had time I would wade into the mess to see if I could hack it to make it work. Just as a side note, I noted that DarwinPorts had an up to date Qt port and MySQL driver, but no krecipes port - see my note above about wasted effort.
Anyway, I found that the plugin needs
CONFIG += thread
when it's compiled. So I extracted the Qt source back into /sw/lib/qt3/src with the aim of just compiling the driver again and over-writing the one that was installed by the port. Compiling was relatively easy once instructions were found, the thread directive from above added, and some questionable advice was also found on how to get rid of a compile error I was getting about the "sql" module not being available. :)
It sounds so easy, but you have no idea how long it took to work all this out. Good news is that krecipes now works locally for me, but the bad news is that I now have to remember this custom hack as the next upgrade of Qt-MySQL driver within Fink is probably going to break things all over again.
Please send me a link if anyone has found a good page that summarises and reviews the current Mac ports and packaging systems...
And so I needed to venture into the mess that is the current Mac open-source ports/packaging system. I say "system" (singular) deliberately; I well know there are many options available in this space, but there really shouldn't be. Good operating systems tend to really only have one system that everyone uses to install and manage open-source software so that effort is not duplicated unnecessarily (which is exponential due to the thousands of applications that need to be tracked in any packaging system). Any gripes with how the current system works should be sorted out within that system and let it evolve over time rather than splitting off yet another effort to do ports/packages "a better way".
Apart from the aforementioned waste of effort by port maintainers, it's just down right confusing for users (even experienced ones). The Mac is a perfect example. It has one of the best application management systems from a end-user point of view (drag and drop to install and delete - how easy is that!) and any system that provides access to open-source software should try and be that easy as well. But if you want to tread in this space you are met with a bewildering array of options; Fink, DarwinPorts, MacPorts, RPM on MacOS X, and it wouldn't surprise me if there are others as well.
I think there may be some link between DarwinPorts and MacPorts, but quite frankly, that proves my point about how confusing it is. These sites exist, various people advise using them for different reasons, and there really isn't any definitive advice on which one you should use. The web sites for these systems are universally bad as well - there just isn't clean information on any of them about why you should use their system and what the key differences are between the systems. Unless you want to spend hours and hours researching the history of these systems you're really left in the dark. And quite frankly the whole point of these systems is to save you hours and hours and not force you to waste hours and hours.
Ok, rant finished. :) For some reason in the past I had actually installed Fink on my test Mac, so I started there. As it turns out, this is the only system which has a port for krecipes anyway... *sigh* Unfortunately, it's been setup to compile and link against an older version of Qt (3.3) that compiles as multi-threaded (good), but the sub-port for the MySQL plugin compiles as single-threaded (bad). So krecipes doesn't work and spits out the following error message as expected:
Conflict in /sw/lib/qt3/plugins/sqldrivers/libqsqlmysql.so:
Plugin uses single threaded Qt library!
After no response from the port maintainer in 1-2 months (what a surprise) I decided if I had time I would wade into the mess to see if I could hack it to make it work. Just as a side note, I noted that DarwinPorts had an up to date Qt port and MySQL driver, but no krecipes port - see my note above about wasted effort.
Anyway, I found that the plugin needs
CONFIG += thread
when it's compiled. So I extracted the Qt source back into /sw/lib/qt3/src with the aim of just compiling the driver again and over-writing the one that was installed by the port. Compiling was relatively easy once instructions were found, the thread directive from above added, and some questionable advice was also found on how to get rid of a compile error I was getting about the "sql" module not being available. :)
It sounds so easy, but you have no idea how long it took to work all this out. Good news is that krecipes now works locally for me, but the bad news is that I now have to remember this custom hack as the next upgrade of Qt-MySQL driver within Fink is probably going to break things all over again.
Please send me a link if anyone has found a good page that summarises and reviews the current Mac ports and packaging systems...
Thursday, 7 February 2008
OpenVPN on OS X
For some reason, most VPNs are approached from a user-level rather than a machine-level point of view. That is, most end-user VPN software assumes that a user has their own authentication mechanism (key, token, etc) and that they establish the virtual network over some other possibly untrusted network when they want to "connect".
But what about the concept of a machine that connects over a VPN? One that either connects on startup or when required and then all users have access to the VPN for certain traffic. It strikes me as odd that this isn't a more common requirement, as I think it has lots of uses, even to the end user. So it's a little disappointing that while you currently can set up machine-level VPNs it normally requires quite a bit of low level buggering around and tinkering. Just because I'm capable of it doesn't mean that I want to do it all the time...
Anyway, I've worked out some hacks that suit my purpose under OS X. They are available here for the enjoyment of all. Please let me know if you've got any feedback, improvements, etc. I'd love to hear from you.
But what about the concept of a machine that connects over a VPN? One that either connects on startup or when required and then all users have access to the VPN for certain traffic. It strikes me as odd that this isn't a more common requirement, as I think it has lots of uses, even to the end user. So it's a little disappointing that while you currently can set up machine-level VPNs it normally requires quite a bit of low level buggering around and tinkering. Just because I'm capable of it doesn't mean that I want to do it all the time...
Anyway, I've worked out some hacks that suit my purpose under OS X. They are available here for the enjoyment of all. Please let me know if you've got any feedback, improvements, etc. I'd love to hear from you.
Tuesday, 29 January 2008
OS X file locking over NFS
On my home network I have an old FreeBSD-based machine used as a central file server. It's used to share various directories via all manner of protocols such as NFS and SMB/CIFS; and importantly, home directories via NFS where possible.
For a while now though, I didn't have file locking working from my Mac as allowing any rpcbind allocated ports through firewalls is of course a massive pain. I just lived without. This had the unfortunate side effect though that occasionally an application would refuse to unzip/unarchive properly unless I moved it to a local drive first, and worse, most sqlitedb based applications not working quite right (or not at all.)
Thankfully, the latest 6.3 release of FreeBSD finally allows static port allocation of rpc.statd and rpc.lockd in addition to mountd (which has been there for a while.) So after upgrading my file server I eagerly locked these ports down, finally allowed the appropriate holes through the firewall, and rebooted my Mac as the final test.
And all seems to be good. :) It's early days though. But my quick test passed with flying colours (installed and fired up Miro and it worked - I've had a bug logged in their tracking system for a while regarding install problems on remotely mounted directories.)
So, fingers crossed that all the other apps I've had to work around by sym' linking out to a local directory (like SyncServices...) will now start working as well once I move them back...
For a while now though, I didn't have file locking working from my Mac as allowing any rpcbind allocated ports through firewalls is of course a massive pain. I just lived without. This had the unfortunate side effect though that occasionally an application would refuse to unzip/unarchive properly unless I moved it to a local drive first, and worse, most sqlitedb based applications not working quite right (or not at all.)
Thankfully, the latest 6.3 release of FreeBSD finally allows static port allocation of rpc.statd and rpc.lockd in addition to mountd (which has been there for a while.) So after upgrading my file server I eagerly locked these ports down, finally allowed the appropriate holes through the firewall, and rebooted my Mac as the final test.
And all seems to be good. :) It's early days though. But my quick test passed with flying colours (installed and fired up Miro and it worked - I've had a bug logged in their tracking system for a while regarding install problems on remotely mounted directories.)
So, fingers crossed that all the other apps I've had to work around by sym' linking out to a local directory (like SyncServices...) will now start working as well once I move them back...
Thursday, 6 December 2007
Windows Installer and Encrypted Directories
Since I've been forced to use my XP laptop more recently, I decided to re-install VersionTracker to keep me notified of any important updates. As I have come to expect, this failed. Even better, it failed in a fairly similar way to the iTunes problems I was having in my last post. An error message of "The system cannot open the device or file specified" and an Event Log ID of 10005.
What on earth is going on here? Deciding to now do a more general search on this issue, I fairly quickly came across the following Microsoft Knowledge Base article titled, 'You receive a "cannot open the device or file specified" error message when you try to install a Windows Installer package.'
Ay, caramba! I would have thought a better fix would have been to correct the fairly fundamental flaw in Windows Installer than to try and bugger around with removing encryption from various directories. After all, a good standard install of XP should have all home directories encrypted for each user to prevent access from other users and people with physical access to your computer (especially if you've got a laptop like I have.)
But seeing how this seems to be disrupting more and more installs lately, I decided to make my TEMP and TMP directories unencrypted with the assumption that important data shouldn't be living in there anyway, and the directories should be regularly cleaned up to not leave data lying around there for long.
But as the KB article also suggests, it appears that a lot of the time that the directory you are running the installer from has to be unencrypted as well. *sigh* There goes installing from my desktop from now on. I have to download to my temporary (unencrypted) directory and then install from there. Oh, to have the cute little "Downloads" folder pop-up that gives quick, one-click access to recent downloads that was in the recent OS X "Leopard" release on XP...
This worked for VersionTracker. I then decided to try and do a bit of other upgrading to see if I could stumble across any other issues. I did. There was an update available for Adobe Reader and lo and behold, it downloaded and failed to update correctly. Digging behind the scenes (now I knew what to look for), I found that it was downloading to
C:\Documents and Settings\lee\Local Settings\Application Data\Adobe\Updater5\Install
*sigh* again. This directory is obviously encrypted (as is every directory under my home by default) as well. Luckily I noticed that the Adobe updater has an option to configure where it downloads its updates to, so I pointed that out to a suitable unencrypted directory and now it works as well.
But it's inevitable that I am going to keep having issues with Windows Installer files until they fix this brain damaged flaw. Each and every application that decides it wants to use it's own directory for automatic updates will trigger off this problem that I will need to go in and fix by hand. I guess I should be grateful that at least I know how to recognise the problem now and how to quickly fix it. Hopefully this post might help someone else as well.
I hate Windows. I hate stupid, lazy programmers even more.
I do like gardens however.
What on earth is going on here? Deciding to now do a more general search on this issue, I fairly quickly came across the following Microsoft Knowledge Base article titled, 'You receive a "cannot open the device or file specified" error message when you try to install a Windows Installer package.'
Ay, caramba! I would have thought a better fix would have been to correct the fairly fundamental flaw in Windows Installer than to try and bugger around with removing encryption from various directories. After all, a good standard install of XP should have all home directories encrypted for each user to prevent access from other users and people with physical access to your computer (especially if you've got a laptop like I have.)
But seeing how this seems to be disrupting more and more installs lately, I decided to make my TEMP and TMP directories unencrypted with the assumption that important data shouldn't be living in there anyway, and the directories should be regularly cleaned up to not leave data lying around there for long.
But as the KB article also suggests, it appears that a lot of the time that the directory you are running the installer from has to be unencrypted as well. *sigh* There goes installing from my desktop from now on. I have to download to my temporary (unencrypted) directory and then install from there. Oh, to have the cute little "Downloads" folder pop-up that gives quick, one-click access to recent downloads that was in the recent OS X "Leopard" release on XP...
This worked for VersionTracker. I then decided to try and do a bit of other upgrading to see if I could stumble across any other issues. I did. There was an update available for Adobe Reader and lo and behold, it downloaded and failed to update correctly. Digging behind the scenes (now I knew what to look for), I found that it was downloading to
C:\Documents and Settings\lee\Local Settings\Application Data\Adobe\Updater5\Install
*sigh* again. This directory is obviously encrypted (as is every directory under my home by default) as well. Luckily I noticed that the Adobe updater has an option to configure where it downloads its updates to, so I pointed that out to a suitable unencrypted directory and now it works as well.
But it's inevitable that I am going to keep having issues with Windows Installer files until they fix this brain damaged flaw. Each and every application that decides it wants to use it's own directory for automatic updates will trigger off this problem that I will need to go in and fix by hand. I guess I should be grateful that at least I know how to recognise the problem now and how to quickly fix it. Hopefully this post might help someone else as well.
I hate Windows. I hate stupid, lazy programmers even more.
I do like gardens however.
Tuesday, 4 December 2007
iTunes 7.5 on XP upgrade issues
Never one to make life simple for myself, I decided to do a bit of upgrading on my laptop today.
The Apple Software Update service had helpfully informed me that the latest iTunes + Quicktime updates (7.5.0.20 and 7.3.0.70 respectively) were available to download, so I decided to do the download through this tool instead of just using it as a notification service and doing a full, manual download by hand like I normally do. I'm pretty sure the MSI packages that are downloaded through this tool are pretty much full installs, so it's not really saving any bandwidth to do it this way (which is a shame - it would be worth doing if they were actually patches/deltas).
Of course the update failed. No useful message as to why of course. Just an extremely unhelpful pop-up saying "The system cannot open the device or file specified," and an equally unhelpful Application Event Log message (Event ID of 1013 and that's about it.) It appeared as if the failure was happening during "Starting Apple Mobile Device Support installation..."
Apple Software Update suggested I should try and install the MSI by hand, so that is what I then dutifully tried to do. I tried the iTunes MSI file and this failed as well, with no further useful information. Trying the AppleMobileDeviceSupport MSI file failed with the same error message, but with an error code as well (2755). The Application Event Log entry was more verbose, but equally as unhelpful except for a new Event ID of 10005. Seriously, would it kill programmers to actually log why they're putting an entry into the Event Log in the first place (and not just the what)?!
A quick Google on the various error messages and Event IDs didn't really turn up anything useful at all, except for vague suggestions of cleaning up the MSI Installer database through use of the "Windows Installer CleanUp Utility" and deleting the entry for the application that wasn't installing properly. This seemed worth a try if the application in question doesn't require too much custom configuration (which applied in my case as I just needed to adjust the install path like I do for all my installs anyway.)
So the next issue I ran into is that this tool needs Windows Script Host enabled to use it. Somewhere along the way, I've disabled that completely on my laptop due to the inherent security weaknesses it introduces and the fact that nothing I use needs it. Until now of course. So I temporarily re-enabled it, which basically just means setting the following registry key back to "1":
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows Script Host\Settings\Enabled
I removed the entries for both iTunes and Apple Mobile Device Support, and attempted to the re-do the Apple Mobile Device Support install again via the MSI. Same problem. Same problem with iTunes.
I then decided to delete iTunes completely and try a fresh install, except when I went to Add/Remove programs, it wasn't there... which on reflection is probably obvious as the Windows Installer Cleanup Utility most likely removes the entry that is used by Add/Remove Programs to track what software is installed. Mental note for the future.
At this stage, I began to suspect that my installation of Quicktime Alternative might have something to do with things, although I was under the impression that an upgrade of Quicktime should pretty much install straight over the top without too much hassle and I'd only be left with the hassle of disabling Quicktime and getting Quicktime Alternative to work again. I had installed Quicktime Alternative so that MPEG Streamclip would work properly on my laptop when I forced to use it instead of one of my Macs.
So the next step was to uninstall Quicktime Alternative to see if this made a difference. After the reboot (grr...) that this required, I then retried the iTunes MSI install and this failed as before. I now tried the Quicktime only MSI install and this failed in the same way as the Apple Mobile Device Support install (error code 2755 and Event ID 10005.) Which I thought was interesting, as my guess was that they were both looking for some file and/or service that wasn't present or running. Why couldn't the programmer just log that?! Without that info, I was either forced to spend even more frustrating time trying to debug/deduce what was missing through process tracing tools, or just use the de-facto, brute force fix on almost all things Windows related - do a complete reinstall.
This meant going back to the full setup files for iTunes + Quicktime instead of just the updated MSI files. I tried an old Quicktime full install that I had (7.2.0.240) and it worked! I then tried the newer MSI file... and it failed. Wow. By this stage though, the call of the garden was strong, and I decided to admit defeat and download the full installs of both iTunes + Quicktime and reinstall by hand. Bugger using the updates; I'll just have to stomach paying for the redundant downloads.
Which of course worked. Sigh. If there's one thing that I hate worse than the atrocious logging used by the vast majority of lazy programmers that makes life far more difficult for system administrators and support personnel that it should be, it's brain-dead, poorly tested upgrades to software that almost seem to rely upon the trained response in Windows users to just reinstall everything from scratch whenever something goes wrong.
Deep breath. I disabled WSH again, and then started to muse over how well my iPod Nano would stand up to direct sunlight and the occasional bit of dirt thrown at it...
The Apple Software Update service had helpfully informed me that the latest iTunes + Quicktime updates (7.5.0.20 and 7.3.0.70 respectively) were available to download, so I decided to do the download through this tool instead of just using it as a notification service and doing a full, manual download by hand like I normally do. I'm pretty sure the MSI packages that are downloaded through this tool are pretty much full installs, so it's not really saving any bandwidth to do it this way (which is a shame - it would be worth doing if they were actually patches/deltas).
Of course the update failed. No useful message as to why of course. Just an extremely unhelpful pop-up saying "The system cannot open the device or file specified," and an equally unhelpful Application Event Log message (Event ID of 1013 and that's about it.) It appeared as if the failure was happening during "Starting Apple Mobile Device Support installation..."
Apple Software Update suggested I should try and install the MSI by hand, so that is what I then dutifully tried to do. I tried the iTunes MSI file and this failed as well, with no further useful information. Trying the AppleMobileDeviceSupport MSI file failed with the same error message, but with an error code as well (2755). The Application Event Log entry was more verbose, but equally as unhelpful except for a new Event ID of 10005. Seriously, would it kill programmers to actually log why they're putting an entry into the Event Log in the first place (and not just the what)?!
A quick Google on the various error messages and Event IDs didn't really turn up anything useful at all, except for vague suggestions of cleaning up the MSI Installer database through use of the "Windows Installer CleanUp Utility" and deleting the entry for the application that wasn't installing properly. This seemed worth a try if the application in question doesn't require too much custom configuration (which applied in my case as I just needed to adjust the install path like I do for all my installs anyway.)
So the next issue I ran into is that this tool needs Windows Script Host enabled to use it. Somewhere along the way, I've disabled that completely on my laptop due to the inherent security weaknesses it introduces and the fact that nothing I use needs it. Until now of course. So I temporarily re-enabled it, which basically just means setting the following registry key back to "1":
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows Script Host\Settings\Enabled
I removed the entries for both iTunes and Apple Mobile Device Support, and attempted to the re-do the Apple Mobile Device Support install again via the MSI. Same problem. Same problem with iTunes.
I then decided to delete iTunes completely and try a fresh install, except when I went to Add/Remove programs, it wasn't there... which on reflection is probably obvious as the Windows Installer Cleanup Utility most likely removes the entry that is used by Add/Remove Programs to track what software is installed. Mental note for the future.
At this stage, I began to suspect that my installation of Quicktime Alternative might have something to do with things, although I was under the impression that an upgrade of Quicktime should pretty much install straight over the top without too much hassle and I'd only be left with the hassle of disabling Quicktime and getting Quicktime Alternative to work again. I had installed Quicktime Alternative so that MPEG Streamclip would work properly on my laptop when I forced to use it instead of one of my Macs.
So the next step was to uninstall Quicktime Alternative to see if this made a difference. After the reboot (grr...) that this required, I then retried the iTunes MSI install and this failed as before. I now tried the Quicktime only MSI install and this failed in the same way as the Apple Mobile Device Support install (error code 2755 and Event ID 10005.) Which I thought was interesting, as my guess was that they were both looking for some file and/or service that wasn't present or running. Why couldn't the programmer just log that?! Without that info, I was either forced to spend even more frustrating time trying to debug/deduce what was missing through process tracing tools, or just use the de-facto, brute force fix on almost all things Windows related - do a complete reinstall.
This meant going back to the full setup files for iTunes + Quicktime instead of just the updated MSI files. I tried an old Quicktime full install that I had (7.2.0.240) and it worked! I then tried the newer MSI file... and it failed. Wow. By this stage though, the call of the garden was strong, and I decided to admit defeat and download the full installs of both iTunes + Quicktime and reinstall by hand. Bugger using the updates; I'll just have to stomach paying for the redundant downloads.
Which of course worked. Sigh. If there's one thing that I hate worse than the atrocious logging used by the vast majority of lazy programmers that makes life far more difficult for system administrators and support personnel that it should be, it's brain-dead, poorly tested upgrades to software that almost seem to rely upon the trained response in Windows users to just reinstall everything from scratch whenever something goes wrong.
Deep breath. I disabled WSH again, and then started to muse over how well my iPod Nano would stand up to direct sunlight and the occasional bit of dirt thrown at it...
Saturday, 24 November 2007
Proxy Woes With Tiger
For a while now, I've been having an intermittent (and therefore highly annoying) problem with proxy settings with a couple of my Mac machines (10.4.x). Every now and then, certain applications seem to ignore the proxy settings I have defined and attempt to directly connect to the Internet.
I know this because I see the packets bounce off my internal firewall. When this happens, I recheck the proxy settings, and even confirm that they work by firing up Software Update and Safari and see if they can connect. They always do. Even going to the command line and using "scutil --proxy" works as expected.
One of the main culprits is VersionTracker Pro. Launching it starts off a disk scan to find applications that might need updating. It does this by submitting the versions on your machine to be compared with a master list located on the VT web site. This always works. You then get a list of applications that can be updated, and if you click on any of them a preview pane gives you some further info on the application and the update available.
Well, it's supposed to. Most of the time this doesn't work for me, as it tries to make the request for this info directly to the Internet instead of going through the proxy. Sometimes however it does, and up pops the details as expected. This could have been put down to flakiness within the VT application itself (and it certainly might be contributing in some way to the problem), but I've also noticed other flakey behaviour with a couple of other applications on my network; online TV guide updates within various apps spring to mind. Oh, and trying to register tools like Mac Pilot.
Anyway, I've lived with this annoyance for quite a while but recently I decided to do something about it once and for all. I tried poking around the lower level workings of Mac OS X to see if something was up there, but as mentioned above, command line utilities like scutil gave me the results I was expecting. I even tried posting to the online Apple forums, but so far, no joy there.
I did start to notice that perhaps there was some differentiation with the requests due to whether HTTP or HTTP/S was used for the request. In the VT example the first request (that works) is HTTP/S and the second request (that fails) is HTTP.
But there seemed to be no difference in my HTTP settings compared to my HTTP/S settings, so eventually I got sick of trying to nut this OS X flakiness out. And I certainly did not want to just blindly pick holes in my firewall just to let some rogue application and/or OS X flakiness have its way. Then it struck me that I should just transparently proxy these requests and be done with it. I already had a FreeBSD-based firewall running Squid for all web access.
Which worked a treat. I still get to enforce only specific rogue application/sites that are allowed to be transparently proxied, and everything is now working as expected. The rogue applications think they are directly communicating with their sites in question, but I still get to log them through the proxy and leave my firewall intact. It's fairly low maintenance to add a new rule if I ever need to. I just need to add forwarding rules to my firewall such as:
Configuring squid to work as a transparent proxy is a doddle nowadays as well. I found a good description at Leslie Viljoen's wiki called Eclectica, which also happens to run on a Mac Mini (a good sign I thought.) His instructions are for Squid 3 (in beta), but as pointed out by a visitor, they also happily work with the later 2.6.x versions as well (I'm running 2.6.16 myself). In my existing config pretty much all I needed to do was add the "transparent" option to the http_port directive and that was it!
So simple at the end of the day, even though initially I didn't think of it because it offended my sensibilities somewhat that things weren't working as they should. I wanted the proxies to work as advertised! But I guess, like a lot of things in life, sometimes you just have to forget about looking at the trees if you want to enjoy the forest...
I know this because I see the packets bounce off my internal firewall. When this happens, I recheck the proxy settings, and even confirm that they work by firing up Software Update and Safari and see if they can connect. They always do. Even going to the command line and using "scutil --proxy" works as expected.
One of the main culprits is VersionTracker Pro. Launching it starts off a disk scan to find applications that might need updating. It does this by submitting the versions on your machine to be compared with a master list located on the VT web site. This always works. You then get a list of applications that can be updated, and if you click on any of them a preview pane gives you some further info on the application and the update available.
Well, it's supposed to. Most of the time this doesn't work for me, as it tries to make the request for this info directly to the Internet instead of going through the proxy. Sometimes however it does, and up pops the details as expected. This could have been put down to flakiness within the VT application itself (and it certainly might be contributing in some way to the problem), but I've also noticed other flakey behaviour with a couple of other applications on my network; online TV guide updates within various apps spring to mind. Oh, and trying to register tools like Mac Pilot.
Anyway, I've lived with this annoyance for quite a while but recently I decided to do something about it once and for all. I tried poking around the lower level workings of Mac OS X to see if something was up there, but as mentioned above, command line utilities like scutil gave me the results I was expecting. I even tried posting to the online Apple forums, but so far, no joy there.
I did start to notice that perhaps there was some differentiation with the requests due to whether HTTP or HTTP/S was used for the request. In the VT example the first request (that works) is HTTP/S and the second request (that fails) is HTTP.
But there seemed to be no difference in my HTTP settings compared to my HTTP/S settings, so eventually I got sick of trying to nut this OS X flakiness out. And I certainly did not want to just blindly pick holes in my firewall just to let some rogue application and/or OS X flakiness have its way. Then it struck me that I should just transparently proxy these requests and be done with it. I already had a FreeBSD-based firewall running Squid for all web access.
Which worked a treat. I still get to enforce only specific rogue application/sites that are allowed to be transparently proxied, and everything is now working as expected. The rogue applications think they are directly communicating with their sites in question, but I still get to log them through the proxy and leave my firewall intact. It's fairly low maintenance to add a new rule if I ever need to. I just need to add forwarding rules to my firewall such as:
# ipfw add <ruleno> fwd <proxy-ip>,<proxy-port> tcp from <client-ip> to <rogue-site-ip> dst-port 80 setup keep-state
Configuring squid to work as a transparent proxy is a doddle nowadays as well. I found a good description at Leslie Viljoen's wiki called Eclectica, which also happens to run on a Mac Mini (a good sign I thought.) His instructions are for Squid 3 (in beta), but as pointed out by a visitor, they also happily work with the later 2.6.x versions as well (I'm running 2.6.16 myself). In my existing config pretty much all I needed to do was add the "transparent" option to the http_port directive and that was it!
So simple at the end of the day, even though initially I didn't think of it because it offended my sensibilities somewhat that things weren't working as they should. I wanted the proxies to work as advertised! But I guess, like a lot of things in life, sometimes you just have to forget about looking at the trees if you want to enjoy the forest...
Subscribe to:
Posts (Atom)