Anyone can think of myriad reasons to run a Tor hidden service. Surely many unsavoury endeavours spring to mind but of course, there are as many noble ones. There are also various pragmatic causes like circumventing lousy NATs. Me? I just wanted to play around with my router.

Configuring a hidden service is actually straightforward so to make things more interesting, this article will cover configuring a hidden service on a Turris Omnia router with the help of Linux Containers to maximise isolation of components. While some steps will be Omnia-specific, most translate easily to other systems, so this post may be applicable regardless of the distribution used.

Do you recall when I decided to abuse Go’s runtime and play with string↔[]byte conversion? Fun times… I wonder if we could do the same to Java?

To remind ourselves of the ‘problem’, strings in Java are immutable but because Java has no concept of ownership or const keyword to make true on that promise, Java runtime has to make a defensive copy each time a new string is created or when string’s characters are returned.

Alas, do not despair! There is another way (exception handling elided for brevity):

private static Field getValueField() {
	final Field field = String.class.getDeclaredField("value");
	field.setAccessible(true);
	/* Test that it works. */
	final char[] chars = new char[]{'F', 'o', 'o'};
	final String string = new String();
	field.set(string, chars);
	if (string.equals("Foo") && field.get(string) == chars) {
		return field;
	}
	throw new UnsupportedOperationException(
		"UnsafeString not supported by the runtime");
}

private final static Field valueField = getValueField();

public static String fromChars(final char[] chars) {
	final String string = new String();
	valueField.set(string, chars);
	return string;
}

public static char[] toChars(final String string) {
	return (char[]) valueField.get(string);
}

However. There is a twist…

I’ve recently found myself in need of an RGB↔XYZ transformation matrix expressed to the maximum possible precision. Sources on the Internet typically limit the precision to a handful decimal places so I’ve performed do the calculations myself.

What we’re looking for is a 3-by-3 matrix \(M\) which, when multiplied by red, green and blue coordinates of a colour, produces its XYZ coordinates. In other words, a change of basis matrix from a space whose basis vectors are RGB’s primary colours: $$ M = \begin{bmatrix} X_r & X_g & X_b \\ Y_r & Y_g & Y_b \\ Z_r & Z_g & Z_b \end{bmatrix} $$

In an earlier entry I’ve changed generated key file used for disk encryption from 4096 to meagre 64 bytes. I gave no mention of that adjustment considering it unimportant but have since been enquired about security of such a short password.

Rest assured, a 64-byte key file is sufficient for any symmetric encryption (disk encryption being one example) and anything more does not improve security.

Yes… I’ve started coding in Go recently. It lacks many things but the one feature relevant to this post is const keyword. Arrays and slices in particular are always mutable and so equivalent of C’s const char * does not exist.

On the other hand, strings are immutable which means that conversion between a string and []byte requires memory allocation and copying of the data¹. Often this might be acceptable but to squeeze every last cycle the following two functions might help achieve zero-copy implementation:

func String(bytes []byte) string {
	hdr := *(*reflect.SliceHeader)(unsafe.Pointer(&bytes))
	return *(*string)(unsafe.Pointer(&reflect.StringHeader{
		Data: hdr.Data,
		Len:  hdr.Len,
	}))
}

func Bytes(str string) []byte {
	hdr := *(*reflect.StringHeader)(unsafe.Pointer(&str))
	return *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
		Data: hdr.Data,
		Len:  hdr.Len,
		Cap:  hdr.Len,
	}))
}

Depending on the length of the strings, the difference in performance might be noticeable:

I’ve been reading tutorials on using key-files for disk encryption. Common approach for generating such files is to create it using something similar to head -c 4096 /dev/urandom >key-file and only then change it’s permissions (usually with a plain chmod 400 key-file) to prevent others from reading it.

Please, stop doing this and spreading that method. The correct way of achieving the effect is:

(umask 077; head -c 64 /dev/random >key-file)

Or if the file needs to be created as root while command is run by a different user:

sudo sh -c 'umask 077; head -c 64 /dev/random >key-file'

The first method creates the file as world-readable¹ and before its permission are changed anyone can read it. The second method creates the file as readable only by its owner from the beginning thus preventing the secret disclosure.

A well known way of generating random floating-point numbers in the presence of a pseudo-random number generator (PRNG) is to divide output of the latter by one plus its maximum possible return value.

extern uint64_t random_uint64(void);

double random_double(void) {
	return random_uint64() / (UINT64_MAX + 1.0);
}

This method is simple, effective, inefficient and wrong on a few levels.

Python! My old nemesis, we meet again. Actually, we meet all the time, but despite that there are always things which I cannot quite remember how to do and need to look them up. To help with the searching, here there are collected in one post:

• Converting a date into a timestamp
• Re-rising Python exception preserving back-trace
• Flattening a list in Python

Below is a list of materials about Contiguous Memory Allocator (CMA) and topics relating to it which may be of interest.

Michał Nazarewicz and Marek Szyprowski. 2012. Continuous Memory Allocator, version 24.

The final patchset that was merged in Linux 3.5.

Michał Nazarewicz. 2013. Alokacja ciągłych fizycznie obszarów pamięci w systemie Linux. Bachelor’s thesis. WEiTI/ISE, PW, Warsaw.

🇵🇱 Diploma thesis in Polish on the Continuous Memory Allocator.

Michał Nazarewicz. 2012. A Deep Dive into CMA. Linux Weekly News (March 2012).

A description of the way to integrate CMA with an architecture as well as short summary of how exactly CMA works.

Michał Nazarewicz. 2012. Deep Dive into Contiguous Memory Allocator.

A description of how to use and integrate CMA with an architecture. It is a first part of an extended version of the above LWN article and as such it includes much more details.

Michał Nazarewicz. 2012. Contiguous Memory Allocator: Allocating Big Chunks of Physically Contiguous Memory. LinuxCon Europe, Barcelona, Spain.

The presentation from the LinuxCon Europe (LCE) 2012 about CMA.

Barry Song. 2012. A Simple Kernel Module as a Helper to Test CMA, vrsien 4.

A short and simple driver that can be used to test CMA as well as see how it is used.

Jonathan Corbet. 2011. A Reworked Contiguous Memory Allocator. Linux Weekly News (June 2011).

An overwiev of the Contiguous Memory Allocator.

Jonathan Corbet. 2011. CMA and ARM. Linux Weekly News (June 2011).

An overview of the linear mapping problems CMA had on ARM platforms, and why the early fixups are required.

Laura Abbott. 2012. Revoke LRU when trying to drop buffers.

Patch which tries to improve CMA’s performance by removing buffer from LRU prior to migration. The thread also mentiones problem with ext4 not supporting migration of journal pages.

Jonathan Corbet. 2010. Memory Compaction. Linux Weekly News (January 2010).

An overview of Mel Gorman’s compaction patches. Compaction code is used by CMA for scanning for and migrating non-free pages.

Jonathan Corbet. 2009. Transcendent memory. Linux Weekly News (July 2009).

Overwiev of an idea behind and implementation of the transcendent memory. Such memory can be marked ‘ephemeral’ which means that kernel can discard it if it wishes to.

Jonathan Corbet. 2011. POSIX_FADV_VOLATILE. Linux Weekly News (November 2011).

An overwiev of John Stultz’s POSIX_FADV_VOLATILE implementation which is one of the things that CMA work with nicely.

Minchan Kim. 2012. Discard clean pages during contiguous allocation instead of migration.

Patch changing CMA so that clean pages are discarded instead of migrating which improves CMA’s performance.

There are a lot of misconceptions about Unicode. Most are there because people assume what they know about ASCII or ISO-8859-* is true about Unicode. They are usually harmless but they tend to creep into minds of people who work with text which leads to badly designed software and technical decisions made based on false information.

Without further ado, here’s a few facts about Unicode that might surprise you.