Below is a list of materials about Contiguous Memory Allocator (CMA) and topics relating to it which may be of interest.

Michał Nazarewicz and Marek Szyprowski. 2012. Continuous Memory Allocator, version 24.

The final patchset that was merged in Linux 3.5.

Michał Nazarewicz. 2013. Alokacja ciągłych fizycznie obszarów pamięci w systemie Linux. Bachelor’s thesis. WEiTI/ISE, PW, Warsaw.

🇵🇱 Diploma thesis in Polish on the Continuous Memory Allocator.

Michał Nazarewicz. 2012. A Deep Dive into CMA. Linux Weekly News (March 2012).

A description of the way to integrate CMA with an architecture as well as short summary of how exactly CMA works.

Michał Nazarewicz. 2012. Deep Dive into Contiguous Memory Allocator.

A description of how to use and integrate CMA with an architecture. It is a first part of an extended version of the above LWN article and as such it includes much more details.

Michał Nazarewicz. 2012. Contiguous Memory Allocator: Allocating Big Chunks of Physically Contiguous Memory. LinuxCon Europe, Barcelona, Spain.

The presentation from the LinuxCon Europe (LCE) 2012 about CMA.

Barry Song. 2012. A Simple Kernel Module as a Helper to Test CMA, vrsien 4.

A short and simple driver that can be used to test CMA as well as see how it is used.

Jonathan Corbet. 2011. A Reworked Contiguous Memory Allocator. Linux Weekly News (June 2011).

An overwiev of the Contiguous Memory Allocator.

Jonathan Corbet. 2011. CMA and ARM. Linux Weekly News (June 2011).

An overview of the linear mapping problems CMA had on ARM platforms, and why the early fixups are required.

Laura Abbott. 2012. Revoke LRU when trying to drop buffers.

Patch which tries to improve CMA’s performance by removing buffer from LRU prior to migration. The thread also mentiones problem with ext4 not supporting migration of journal pages.

Jonathan Corbet. 2010. Memory Compaction. Linux Weekly News (January 2010).

An overview of Mel Gorman’s compaction patches. Compaction code is used by CMA for scanning for and migrating non-free pages.

Jonathan Corbet. 2009. Transcendent memory. Linux Weekly News (July 2009).

Overwiev of an idea behind and implementation of the transcendent memory. Such memory can be marked ‘ephemeral’ which means that kernel can discard it if it wishes to.

Jonathan Corbet. 2011. POSIX_FADV_VOLATILE. Linux Weekly News (November 2011).

An overwiev of John Stultz’s POSIX_FADV_VOLATILE implementation which is one of the things that CMA work with nicely.

Minchan Kim. 2012. Discard clean pages during contiguous allocation instead of migration.

Patch changing CMA so that clean pages are discarded instead of migrating which improves CMA’s performance.

There are a lot of misconceptions about Unicode. Most are there because people assume what they know about ASCII or ISO-8859-* is true about Unicode. They are usually harmless but they tend to creep into minds of people who work with text which leads to badly designed software and technical decisions made based on false information.

Without further ado, here’s a few facts about Unicode that might surprise you.

There are multiple ways to customise Bash prompt. There’s no need to look for long to find plethora of examples with fancy, colourful PS1s. What have been a bit problematic is having text on the right of the input line. In this article I’ll try to address that shortcoming.

Getting text on the right

The typical approach is using PROMPT_COMMAND to output desired content. The variable specifies a shell code Bash executes prior to rendering the primary prompt (i.e. PS1).

The idea is to align text to the right and then using carrier return move the cursor back to the beginning of the line where Bash will start rendering its prompt. Let’s look at an example of showing time in various locations:

__command_rprompt() {
	local times= n=$COLUMNS tz
	for tz in ZRH:Europe/Zurich PIT:US/Eastern \
	          MTV:US/Pacific TOK:Asia/Tokyo; do
		[ $n -gt 40 ] || break
		times="$times ${tz%%:*}\e[30;1m:\e[0;36;1m"
		times="$times$(TZ=${tz#*:} date +%H:%M)\e[0m"
		n=$(( $n - 10 ))
	done
	[ -z "$times" ] || printf "%${n}s$times\\r" ''
}
PROMPT_COMMAND=__command_rprompt

Clearing the line on execution

It has one annoying issue. The right text reminds on screen even after executing a command. Typically this is a matter of aesthetic but it also makes copying and pasting session history more convoluted.

A manual solution is to use redraw-current-line readline function (e.g. often bound to C-l). It clears the line and prints the prompt and whatever input has been entered thus far. PROMPT_COMMAND is not executed so the right text does not reappear.

Lack of automation can be addressed with a tiny bit of readline magic and a ~/.inputrc file which deserves much more fame than what it usually gets.

Tricky part is bindind C-m and C-j to two readline functions, redraw-current-line followed by accept-line, which is normally not possible. This limitation can be overcome by binding the key sequences to a different sequence which will be interpreted recursively.

To test that idea it’s enough to execute:

bind '\C-l:redraw-current-line'
bind '\M-\C-j:accept-line'
bind '\C-j:"\C-l\M-\C-j"' '\C-m:"\C-j"'

Making this permanent is as easy as adding the following lines to ~/.inputrc:

$if Bash
    "\C-l": redraw-current-line
    "\e\C-j": accept-line
    "\C-j": "\C-l\e\C-j"
    "\C-m": "\C-l\e\C-j"
$endif

With that, the right prompt will disappear as soon as the shell command is executed. (Note the use of \M- in bind command vs. \e in ~/.inputrc file).

For people who know me it should come with no surprise that I support free software in most forms it can take. I also believe that if someone gives you something at zero price, basic courtesy dictates that you follow wishes of that person. This is why when Software Freedom Conservancy started a GPL Compliance Project for Linux Developers I didn’t hesitate even for a minute to offer little Linux copyright I held to help the effort.

Most importantly though, it is why I fully support Conservancy in taking legal action against VMware which for years has been out of compliance with Linux’s license.

If you care about free software, the GPL or want more projects like OpenWrt, consider donating to help Christoph Hellwig and the Conservancy with their legal battle against this multi-billion-dollar corporation who for some reason decided to free-ride on other people’s work without respecting their wishes.

If you can’t or don’t want to donate, twitting something along the lines of ‘Play by the rules, @VMware. I defend the #GPL with Christoph & @Conservancy. #DTRTvmware Help at https://sfconservancy.org/supporter/’ or otherwise spreading the word will help as well. Oh, and in case you were, like I was, wondering — DTRT stands for ‘do the right thing’.

And if you want to know more:

• Press release
• LWN article

Don’t you hate when you need to do something you had done before, but cannot remember how exactly? I’ve been in that situation several times and sometimes looking up for a correct method turned out considerably harder than it should. To alleviate the need for future Googling, here’s a bag of notes I can reference easily:

• Getting past Chrome certificate error
• ‘Compressing’ git repository
• Changing commits author
• How to start a letter in English
• Summing numbers in a column in Emacs

Looking for Python stuff? Those are now in separate post:

• Converting date to timestamp
• Re-rising Python exception preserving back-trace
• Flattening a list in Python

Outside of functional programming context, map-reduce refers to a technique for processing data. Thanks to properties of map and reduce operations, computations which can be expressed using them can be highly parallelised, which allows for faster processing of high volumes of data.

If you’ve ever wondered how tools such as Apache Hadoop work, you’re at the right page. In this article I’ll explain what map and reduce are and later also introduce a shuffle phase.

Same as my previous article written in Polish, this text will describe some steps I take after installing Slackware Linux. I try to strike a balance between performance, security and usability, but not everything written here may work for everyone. You have been warned.

HTML has gone quite a long way since its inception. This means a lot of new features but also some small incompatibilities which may pose issues in certain situations. For instance, when posting a code snippet for others to include on their websites, it’s best if it works correctly on as many sites as possible which implies being compatible with as many versions of HTML as possible. But how to create a snippet that works both in HTML and XHTML? Here are a few tips:

CSS sprites aren’t anything new. They have been around for years, and are one of the methods to optimise website’s load time. The idea is to incorporate multiple images into one and in this way decrease number of round trips between the server and the browser.

In its traditional use, CSS sprites work as a replacement for images and cannot be used as a background. Alas that is exactly what I wanted to do with a quote and flag icons like the following:

Update: This website has evolved slightly since 2013. The flags are no longer used (replaced by content negotiation) and quote sprite icon has been replaced by an SVG. While I no longer use this technique, it is of course still valid.

After some playing around I’ve finally figured out how to get this working. Even though there are some caveats, sprites can be used as a top-left no-repeat background image as well.

Surprisingly I couldn’t find any HTTPS-aware examples how to drop the www. prefix from web hosts in Apache, so I had to come up with one myself. Firstly, the following lines need to find their way to the end of Apache configuration file (/etc/httpd/conf/httpd.conf or something):

RewriteEngine on
RewriteCond %{HTTPS} off
RewriteCond %{HTTP_HOST} ^www\.(.*)$
RewriteRule ^(.*)$ http://%1$1 [L,R=301]

Secondly, analogous lines need to be added inside of the <VirtualHost _default_:443> directive of mod_ssl configuration file (/etc/httpd/conf.d/ssl.conf or similar), like so:

<VirtualHost _default_:443>
	# … various directives …

	# Here’s what needs to be added:
	RewriteEngine on
	RewriteCond %{HTTP_HOST} ^www\.(.*)$
	RewriteRule ^(.*)$ https://%1$1 [L,R=301]
</VirtualHost>

Now, after a restart, Apache will drop the www. prefix for both secure and insecure connections.