Working With History in Bash

Working With History in Bash: „

Yesterday we talked about favorite bash features (on the ##textmate IRC channel). I figured it was worth posting mine to this blog, they mostly revolve around history, hence the title.

Setup

My shell history collects a lot of complex command invocations which take time to figure out. To ensure that I have access to them at a later time, I have the following 3 lines in my bash init:

export HISTCONTROL=erasedups
export HISTSIZE=10000
shopt -s histappend

The first one will remove duplicates from the history (when a new item is added). For example if you switch between running make and ./a.out in a shell, you may later find that the last 100 or so history items is a mix of these two commands. Not very useful.

The second one increase the history size. With duplicates erased, the history already holds a lot more actual information, but I still like to increase the default size of only 1,000 items.

The third line ensures that when you exit a shell, the history from that session is appended to ~/.bash_history. Without this, you might very well lose the history of entire sessions (rather weird that this is not enabled by default).

History Searching

Now that I have my history preserved nicely in ~/.bash_history there are a few ways to search it.

Using Grep

The most crude is grep. You can do:

history|grep iptables

For me (on this particular Linux server) I get:

4599  iptables -N http-block
4600  iptables -A http-block -s 58.60.43.196 -j DROP
4601  iptables -A INPUT -p tcp --dport 80 -j http-block
4602  iptables -L http-block
4603  iptables-save -c
4604  history|grep iptables

I do this often enough to have an alias for history (which is just h).

From the output I can either copy/paste the stuff I want, or repeat a given history event. You’ll notice that each history event has a number, you can repeat e.g. event number 4603 simply by running:

!4603

I will write a bit more about referencing history events in History Expansion.

Prefix Searching

Similar to how you can press arrow up for the previous history event, there is a function you can invoke for the previous history event with the same prefix as what is to the left of the insertion point.

This function is called history-search-backward and by default does not have a key equivalent. So to actually reach this function, I have the following in ~/.inputrc (or /etc/inputrc when I control the full system):

'\ep': history-search-backward

This places the function on P (escape P). So if I want to repeat the iptables-save -c history event we found in previous section, all I do is type ipt and hit P. If it finds a later event with the same prefix, hit P again to go further back.

This functionality is offered by the readline library, so if you setup this key, you have access to prefix searching in all commands which use this library.

Incremental Search

It is possible to press ⌃R to do an incremental (interactive) search of the history.

Personally I am not a big fan of this feature, so I will leave it at that :)

Update: The reason I dislike ⌃R is both because the interactive stuff just seems to get in the way (when P is what I need 99% of the time) and because it fails in cases where I ‘switch shell’, for example I may do: ssh mm press return, then instantly type: fP and again hit return (to execute free -m on the server called mm). I enter this before the connection to the server has been fully established, and here ⌃R would have been taken by the local shell, but it is the shell history at the server I want to search.

History Expansion

History Expansion was what we did above when we ran !4603. It is a DSL for referencing history events and optionally run transformations on these.

Anyone interested in this should run man bash and search for History Expansion, but just to give you a feel for what it is, I will reference a subset of the manual and provide a few examples.

Event Designators

First, an event designator starts with ! and then the event we want to reference. This can be:

«n»      Reference event number «n».
-«n»     Go «n» events back.
!        Last line (this is the default).
#        Current line.
«text»   Last event starting with «text».
?«text»  Last event containing «text».

So if we want to re-run our iptables-save -c we can do: !ipt.

What’s more useful though is to use history references as part of larger commands.

For example take this example:

% which ruby
/usr/bin/ruby
% ls -l $(!!)
lrwxr-xr-x  1 root  wheel  76 30 Oct  2007 /usr/bin/ruby -> ../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/ruby

Or something like:

% make some_target
(no errors)
% path/to/target some arguments
(no errors)
% !-2 && !-1

Word Designators

In the previous section we referenced entire history events. It is possible to reference just a subset of a history event by appending a : to the event designator and then the word of interest, the two most useful are:

«n»      Reference the «n»’th word.
$        Reference the last word.

So for example we can do:

% mkdir -p /path/to/our/www-files
(no errors)
% chown www:www !$
(no errors)

Here we reference last word of last line. We can also reference stuff on the same line, e.g.:

% cp /path/to/important/file !#:1_backup

To reference the last word of last line one can also press _ which will immediately insert that word.

Modifiers

To make history substitution even more useful (and harder to remember), one can also add a modifier to the event designator.

The most useful modifiers are in my experience :h and :t, these are head and tail respectively or better know as dirname and basename.

An example could be:

% ls -l /path/to/some/file
(listing of file)
% cd !$:h
(change directory to that of file)

Brace Expansion

Somewhat related to the backup example where we reference the first argument as !#:1 and append _backup to this, another approach is bracket expansion.

Anywhere on a command line, one can write {a,b,c} which will expand to the 3 words a, b, and c. If we include a prefix/suffix, that will be part of each of the expanded words. We can also leave the word in the braces empty, and have it expand to just the prefix/suffix, so for example we can do:

% cp /path/to/important/file{,_backup}

This is functionally equivalent to:

% cp /path/to/important/file !#:1_backup

But lack of hardcoded word number is IMO an improvement.

(Via TextMate Blog.)

Leben und Arbeiten mit Social Software und Web 2.0

Anlässlich der tiefgreifenden Veränderungen, die Social Software und Web 2.0 mit sich bringen, hat die MFG Innovationsagentur für IT und Medien des Landes Baden Württemberg die Publikation „a digital lifestyle – leben und arbeiten mit social software“ herausgebracht, die wesentliche Aspekte dieses Wandels beleuchtet.

Auf über 80 Seiten gehen namhafte Autoren aus Wissenschaft und Praxis der Frage nach, wie sich unser Lebensstil und unsere Arbeitswelt in Punkto Kooperation, Interaktion und Partizipation in der digitalen Zukunft entwickeln werden.

Die informative und zugleich interessant gestaltete Publikation wird kostenlos zum Download angeboten und ist zudem als Printversion für eine Schutzgebühr von 15 Euro erwerbbar. Sie eignet sich als Einstieg in das Thema für interessierte Privatpersonen und Unternehmen als Überblick über Möglichkeiten, die das Web 2.0 bietet.

http://www.digital-lifestyle.mfg-innovation.de/?page_id=34

How To Run Firefox 2 and 3 Simultaneously

How To Run Firefox 2 and 3 Simultaneously: „

Firefox 3 was released just yesterday, bringing a wealth of new features to be excited about. From faster launch time and better memory management to interface polish and glitz, and over 8 million downloads already, Firefox 3.0 is truly a gem among modern web browsers.

For those that do web development, though, the 3.0 release means another round of website compatibility testing in both the earlier release and now the current one. For the most part, this isn’t a new challenge, but Firefox can be a bit peculiar: Upon launching Firefox, it checks to see if another instance is already running, and if found, brings up a new browser window of the currently running version. So, launching Firefox 2.0 while 3.0 is running results in two 2.0 windows, and vice versa for 3.0. This effectively stops the casual user from simultaneously running both Firefox 2.0 and 3.0 side by side. As an added side effect, when you completely shut down Firefox and launch a different major version, your profile information can get skewed, resulting in sometimes strange bookmark appearance and lots of checking for updates to installed add-ons. With a little tweaking, though, Firefox 2.0 and 3.0 can be convinced to run independently, each with their own bookmarks and add-ons.

Mac OS X Instructions
Running two versions of Firefox under Mac OS X couldn’t be simpler. The freeware application MultiFirefox takes all the guesswork out of the process by automatically detecting Firefox versions in your main Applications folder and presenting you with a list from which to choose the Firefox versions you’d like to launch. Complete with self-updating (thanks to the ubiquitous Sparkle framework), MultiFirefox is the easiest way to pull off this multi-browser stunt.

Windows Instructions
If you already have Firefox (2.0 or 3.0) installed, it’s important to back up your crucial information before getting started. Navigate to C:\Documents and Settings\YourName\Application Data\Mozilla\Firefox and copy your ‘Profiles’ folder somewhere safe.

Once backed up, download Firefox 2 and/or Firefox 3, depending on what you already have installed. Launch each installer and begin setting up Firefox, but make sure to choose Custom settings. When asked for a place to install Firefox, change the path(s) to C:\Program Files\Mozilla Firefox 3\ (or …\Mozilla Firefox 2), instead of just ‘Mozilla Firefox’. In doing so, you’ll place Firefox 2 at C:\Program Files\Mozilla Firefox 2\, and Firefox 3 at C:\Program Files\Mozilla Firefox 3\. Both versions of Firefox should now be in separate folders. (If you already have one version of Firefox installed at C:\Program Files\Mozilla Firefox\, you may choose to leave that where it is and just install the other version in a separate folder, however you’ll have to make a note of which is where, and modify the following instructions accordingly.)

Create a new shortcut to Firefox 2.0 by right-mouse-button-dragging C:\Program Files\Mozilla Firefox 2\firefox.exe to your Desktop (for now), and choosing ‘Create Shortcuts Here’. Name the shortcut ‘Firefox 2″. Repeat for Firefox 3, changing ‘2″ to ‘3″ where appropriate.

You’re almost there, but now we have to tell Firefox to keep the two instances separate. Open the Properties window for the Firefox 3 shortcut, and add ‘ -ProfileManager’ (space key, minus sign, ProfileManager) to the end of the string of text in the Target field. Click OK, then run that same shortcut to bring up Firefox’s Profile Manager. Rename the existing ‘default’ profile to ‘firefox3″, and also create a new ‘firefox2″ profile:

Once profiles are set, exit Firefox completely. With two profiles in place, each version of Firefox can have its own bookmarks, extensions, and other settings, without interfering with each other. Open up the properties windows for both the Firefox 2 and Firefox 3 shortcuts you made. Remove the -ProfileManager addition from the Firefox 3 shortcut Target, replacing it with -no-remote -p firefox3 (space key, minus sign, no-remote, space key, minus sign, space key, firefox3). Add the same to the Firefox 2 shortcut, again changing the 3 to a 2. Both shortcuts should now have the -no-remote option set, as well as -p followed by the designated profile that matches that Firefox’s version. In short, Firefox 3 gets the firefox3 profile, and Firefox 2 gets the firefox2 profile, seen below:

Click OK on both shortcuts to confirm the changes, and finally double-click each to run Firefox 2 and 3 at the same time!

Also, don’t miss LifeHacker’s Power User’s Guide to Firefox 3 for some excellent 3.0 tips and tricks, as well as ArsTechnica’s briefing of what’s to come in Firefox 3.1.

(Via Command-Tab.)

Implementation Focus: MiaCMS

Implementation Focus: MiaCMS:

Introduction: Chad Auld of the MiaCMS projectChad Auld is one of the principal developers of MiaCMS. MiaCMS is a relatively new open source project, but it’s starting out with a solid base from its roots in Mambo CMS. Chad spent about three years on the Mambo team in various roles including Lead Core Developer and Director on the Mambo Foundation Board; he was a member of the Mambo Steering Committee prior to retiring in February of 2008. Recently, Chad joined with other core Mambo developers to create MiaCMS which is now on its second release. Chad’s role on the MiaCMS team for the first two releases has been to help with the rebranding efforts, build the default WYSIWYG editors, implement the YUI Library, build a REST API for content access, and develop enhanced charting for general CMS statistics and poll results.

How does MiaCMS differentiate itself from other CMS projects out there? Why would someone choose MiaCMS over Drupal or Joomla or other well-known apps in this space?

Yes, there are quite a few content management systems to compete with. Luckily we aren’t really new to the game. Our team contributed toward making Mambo the CMS it is today. We will continue building on that same award-winning base with MiaCMS. (As a side note, it’s worth pointing out that Joomla was also initially based on Mambo about three years ago.)

Some of our current features are:

  • Simple Installation
  • WYSIWYG Content Editors
  • RSS Content Syndication
  • Powerful/Extensible 3rd Party Extension System
  • Flexible Site Theming Capabilities
  • Site Search
  • Sitemap Generation
  • REST Enabled Content & Statistics
  • User Management
  • Multilingual Core

MiaCMS will differentiate itself by making standard content management operations even easier and more flexible than they have been in the past. We will cleanup much of the old legacy code and enhance the extensions interface to simplify custom 3rd party extension development. With the 4.7 release the team will drop support for PHP4 to take advantage of the object-oriented capabilities of PHP5. The team plans to continue building close ties to the community and listening to their feedback. The next few releases will focus on building out many of the wishlist items we have already received from the community.

At some point, you and the development team made the decision to build YUI into MiaCMS. What were the factors that guided your decision?

YUI Menu and TabView on MiaCMS.

We based our decision on a number of important factors; maturity, browser support, documentation, support community, functionality, and flexibility. YUI has a large selection of time-tested components and continues to make valuable additions with each release. For us it is important that the selected framework continue maturing and growing right along side of us. We didn’t want to add yet another library to the system and so it was important to be able to replace existing parts of the CMS with canned components and/or have the flexibility to hook into the framework and use it as a building block for custom components. The YUI documentation is first-class. In fact, it represents some of the best documentation I’ve come across for an open source project. Between the user guides, cheatsheets, api browser, examples, and developer videos, you have just about everything you could ask for. Of course, sometimes documents just aren’t enough. Luckily, we’ve found the YUI support group to be a good place to find additional answers. Last but not least is the topic of browser support. While we’d love to support every browser in existence, it simply isn’t possible. But we do our best to test and code for as many as we can. We think Yahoo! has taken the right approach with its Graded Browser Support model.

What components of YUI are used in Mia?

We are currently utilizing the ResetFontsGrids CSS, Dom Collection, Event Utility, Tabview Control, Button Control, Color Picker Control, Rich Text Editor, Animation Utility, Element Utility, Container Family, and Menu Control. Our production releases also make use of the YUI Compressor which we have integrated with our ANT packager to compress all the CSS and JavaScript in the system. The entire YUI library is included in the system so we are hoping our 3rd party developer community will make use of the library as well. Each custom component comes with its own set of unique requirements and we are confident that YUI can meet their needs, help improve their extensions, and reduce the number of 3rd party libraries the system must carry. In the last release we also build a dynamic loader into the system which allows MiaCMS users to decide between serving files from the local YUI library and serving them from the Yahoo hosting service for the advantages its CDN can bring.

YUI Rich Text Editor on MiaCMS.

Where do you see opportunities for deeper YUI integration with MiaCMS in the months ahead?

We’ve still got a lot more planned for YUI. Mia is carrying a fair amount of legacy JavaScript code in the system since its Mambo base was started about 7 years ago. We’ll be rewriting a good chunk of the JavaScript in Mia over the next few releases and utilizing YUI where possible. Users can expect a drastic reduction in inline JavaScript. We also plan to move away from older styles of event handling like coding individual onclick/onmouseover events and instead rely on the YUI Event Utility to subscribe to DOM events and help us create custom events with the application. Future releases will make heavy use of the YUI Dom Collection as well as the Event and Selector utilities.

In addition to the custom JavaScript found in the CMS there are also a number of external JavaScript libraries included to handle specific functions like tooltips, menus, calendaring, etc. A goal for the project will be to reduce the number of external dependencies and rely on YUI where possible. Two such replacements have already been almost fully implemented within the CMS core and we have started to encourage our 3rd party developers to make the switch as well with their custom extensions. In past releases the menu system relied on JSCookMenu and all tabs within the system relied on WebFX Tab Pane. JSCookMenu has now been fully replaced with the YUI Menu Control and the WebFX Tab Pane conversion to YUI TabView is about 98% complete. We are currently in the process of replacing overLib tooltips with the YUI Tooltip Control. We will also soon replace ‘The DHTML Calendar’ with the YUI Calendar Control. It would also be pretty safe to say you’ll eventually find ContextMenus, TreeView, DataSource, DataTable, Connection Manager, and JSON being used within MiaCMS. We recently selected Open Flash Charts, but as the YUI Charts Control matures and evolves out of an experimental state that may also find it’s way into Mia.

Having developed a complex application implementing YUI, what are your thoughts on the state of YUI as a toolkit? What’s working super well at this point? What weaknesses are you hoping the YUI team will address?

YUI is a feature-rich, well designed, state of the art toolkit. The available components cover a wide variety of the common tasks needed for web application development. We have had great success integrating YUI deeply into the MiaCMS core which we will continue to do with each passing release. The community feedback on the YUI-related changes has been very positive so far. Support for the major browsers is top-notch, the components degrade nicely, and performance is solid. Tools like the YUI Compressor and YSlow are also key in helping us take performance to the next level.

Nothing much to complain about. Overall we have been very happy with our selection of the YUI Library. One of the things I really like about jQuery is the powerful CSS style selectors. I am really looking forward to the YUI Selector Utility coming out of beta. We’ll probably start making heavy use of it even before then, but obviously the more stable it is the better. I also see a lot of potential for the experimental Charts component so I’d like to see it polished up with its functionality being continually expanded as well.

What are the next big frontiers for the MiaCMS project as a whole?

Below is the list of roadmap items in no particular order. Some are already being worked on, some are almost complete, and others are in the planning stages.

  • Improved ACL’s (User/Group Permissions)
  • Database Portability
  • LDAP Support
  • OpenID
  • Dublin Core Metadata
  • OAuth
  • N-Level Content Organization (remove the two tier section/category limitation)
  • Content Versioning
  • Multilingual Content Management
  • Writeable REST Interface
  • Multi-Site Management
  • Improved File & Image Management

(Via Yahoo! User Interface Blog.)

Internet Explorer 8: Beta 1 in deutscher Sprache

Internet Explorer 8: Beta 1 in deutscher Sprache: „Microsoft bietet ab sofort die Beta 1 des Internet Explorer 8 auch in deutscher Sprache an. Anfang März 2008 erschien die erste Vorabversion des kommenden Microsoft-Browsers zunächst nur in englischer Sprache. Bis auf die Lokalisierung unterscheiden sich die beiden Beta-Fassungen nicht. Der Browser unterstützt unter anderem CSS 2.1 sowie erste Teile von HTML 5 und mit Hilfe von ‚Web Slices‘ lassen sich einzelne Teile einer Website gesondert über die Favoriten-Leiste abrufen. (MSIE, Microsoft)“

(Via Golem.de.)

Optimizing Page Loading in the Web Browser

Interessanter Artikel, der unter anderem kurz auf die SCRIPT-Tag-Problematik eingeht.

Optimizing Page Loading in the Web Browser: It is well understood that page loading speed in a web browser is limited by the available connection bandwidth. However, it turns out bandwidth is not the only limiting factor and in many cases it is not even the most important one.

[snip]

It turns out that figuring out ‘all the associated resources’ is the hard part of the problem. The browser does not know what resources it should load until it has completely parsed the document. When the browser first receives the HTML text of the document it feeds it to the parser. The parser builds a DOM tree out of the document. When the browser sees an element like <img src> that references an external resource it requests that resources from the network.

Problems start when a document contains references to external scripts, <script src>. Any script can call document.write(). Parsing can’t proceed further before the script is fully loaded and executed and any document.write() output has been inserted into the document text. Since parsing is not proceeding while the script is being loaded no further requests for other resources are made either. This quickly leads to a situation where the script is the only resource loading and connection parallelism does not get exploited at all. A series of script tags essentially loads serially, hugely amplifying effect of the latency.

The situation is made worse by scripts that load additional resources. Since those resources are not known before the script is executed it is critical to load scripts as quickly as possible. The worst case is a script that load more scripts (by using document.write() to write <script> tags), a common pattern in Javascript frameworks and ad scripts.

[snip]

(Via Surfin‘ Safari.)

Safari 3.1

Safari 3.1 ist soweit und lässt sich ab sofort auf der Apple-Seite herunterladen – sowohl für den Mac als auch für Windows-PCs. Im Vorfeld hatte sich bereits anhand von Vorversionen gezeigt, dass die neue Version ein großer Wurf werden könnte.

Nun bestätigt Apple per Pressemitteilung: „Safari baut Webseiten bis zu 1,9 mal so schnell wie der Internet Explorer 7 und bis zu 1,7 mal so schnell wie Firefox 2 auf“. JavaScript sei bis zu sechs Mal schneller als bei anderen Browsern. Apple unterschlägt bei diesen Geschwindigkeitsangaben zwar, dass die Konkurrenz ebenfalls nicht schläft und Betaversionen von Firefox 3 bereits fast an die Geschwindigkeit von Safari herankommen – dennoch hat Apple unseren Tests zufolge derzeit die Nase vorn, sowohl in punkto Gewschwindigkeit als auch bei der Komaptibilität mit aktuellen und kommenden Web-Standards, wie der Acid3-Test beweist. Safari 3.1 unterstützt zudem als erster Browser sowohl Video- und Audio-Tags in HTML 5 als auch CSS Animationen und kommt darüber hinaus mit CSS Web Fonts zurecht. Voraussetzung ist mindestens Mac OS X 10.4.11, das Update ist über die Software-Aktualisierung erhältlich und für Leopard 39 Megabyte groß, der Tiger-Download zählt 49 Megabyte.

Apple Informationen zum Update: http://docs.info.apple.com/article.html?artnum=307467-de

Via macnews.de