Shaun Harker

Chess Engine Bit Hacks

2022-05-09T00:00:00-04:00

I’ve been making my own chess engine both to sharpen up a bit on my C++ skills and because it is useful for a machine learning project I am doing involving chess. In particular I need to get the legal moves as quickly as possible into python to feed in as training data. I saw a warning never to try to write your own chess engine, so obviously I took the bait.

My first version is complete, and gives a big boost, but it still is seriously slow compared to expertly constructed engines. The way they achieve this is based on creating lots of lookup tables and using bit hacks on 64-bit wide words.

Update 2022-06-29: I ended up writing several versions: https://github.com/shaunharker/chessboard expanding upon ideas from what is written below

Bitboards

In chess engines one typically finds 64-bit wide unsigned integer types (uint64_t in C++) being understood as 8x8 bit matrices representing subsets of the 64 squares of the chessboard. We have to pick a convention, so we’ll make the square a8 correspond to the least significant bit, and then proceed left-to-right then top-to-bottom on the chessboard assigning the consecutive powers of two to the squares. In fact, this is precisely the story of the man who asked the king for a reward based on the following procedure: place one grain of rice on the first square a8, twice as many on b8, and so forth, all the way to h1.

A power of two s is identified with a square with s grains on it; more generally, a positive integer 0 <= x < 2**64 is taken to correspond to the unique set of squares that have precisely x grains on them when taken as a whole.

We can generalize this to nxn boards, where we want the total width of bits to be n**2, and allow 0 <= x < 2**(n*n) to represent a subset of an nxn board in the exact same manner.

Visualizing Bitboards

Big-endian digit ordering is the customary way people write integers, but if we want to visualize a 64-bit binary code as a subset of chessboard squares from White’s perspective according to our convention above, we need to reverse the bit order (i.e. make it little-endian digit ordering) and break it into 8 lines. (Or, more generally, n**2-bit broken into n lines.)

Here is python code that performs these contortions for us:

def little_endian(x, n=8):
    rep = bin(x)[2:]
    return ''.join(reversed('0'*(n*n-len(rep)) + rep))

def ascii_bitboard(x, n=8):
    return ''.join([ a+b for (a,b) in
      zip(little_endian(x, n), ['']*7 + ['\n'])])

def html_bitboard(x, n=8) -> str:
    bitstring = little_endian(x, n)
    chessboardstyle = """border-collapse: collapse; margin: 0 auto;"""
    tdstyle = """width: 25px; height: 25px; border: 1px solid #000;"""
    darksquare = """background-color: #d18b47;"""
    lightsquare = """background-color: #ffce9e;"""
    circlestyle = """width: 100%; height: 100%; background-color: black; border-radius: 70%; position: relative; top: 50%; left: 50%; transform: translate(-50%, -50%);"""
    circlediv = f"""<div style="{circlestyle}"></div>"""
    html = f'<div style="text-align: center;"><table style="{chessboardstyle}">'
    for i in range(n):
        html += '<tr>'
        for j in range(n):
            squarestyle = lightsquare if (i + j) % 2 == 0 else darksquare
            contents = circlediv if bitstring[i * n + j] == '1' else ""
            html += f'<td style="{tdstyle+squarestyle}">{contents}</td>'
        html += '</tr>'
    html += '</table></div>'
    return html

Bitboards as subsets of `{0,1,...,n**2-1}`

It is easy to see that an nxn bitboard can be used to represent a subset of set(range(n**2)) (i.e. {0,1,2,...,n**2-1}) by associating the least significant bit to 0, the next to 1, and so on.

To be precise:

def bitboard_from_set(S):
    return sum(2**s for s in S)

S = [2, 5, 8, 33]
x = bitboard_from_set(S)
html_bitboard(x)

logic

Intersection, Union, and Symmetric Difference can be performed with &, |, and ^, respectively. Works out nicely.

ntz (number of trailing zeros)

We’d also like an efficient way to iterate over the elements of a set. One possible way to accomplish this is via a bithack for the number of trailing zeros function ntz. Equivalently, this gives the position of the least significant 1 in the binary representation (where indexing starts at 0).

The main idea to this end is that we can quickly compute the lowest element, remove it, compute the lowest element, remove it, etc. This is better than testing each of the 64 bits in for loop, particularly for bitboards with small population counts.

To isolate the least significant 1 in the binary sequence is easy: the formula ((x^(x-1)) >> 1) + 1 zeros out all 1’s that are not the least significant, leaving us with a power of 2. To finish, we need a quick way to take the base 2 logarithm of a power of two.

This is the basis for the following bithack for the number of trailing zeros (ntz) in order to read off minimal square in a bitboard. Henry Warren’s Hacker’s Delight book gives an algorithm based on De Bruijn sequences for 32-bits, and 64-bit implementations can be found in the wild. Here is the algorithm in python:

def ntz(x):  # for 0 <= x < 2**64
    return [
         0, 47,  1, 56, 48, 27,  2, 60,
        57, 49, 41, 37, 28, 16,  3, 61,
        54, 58, 35, 52, 50, 42, 21, 44,
        38, 32, 29, 23, 17, 11,  4, 62,
        46, 55, 26, 59, 40, 36, 15, 53,
        34, 51, 20, 43, 31, 22, 10, 45,
        25, 39, 14, 33, 19, 30,  9, 24,
        13, 18,  8, 12,  7,  6,  5, 63
        ][(0x03f79d71b4cb0a89*(x^(x-1)))>>58]

Converting from square indexing {0,1,...,63} to bitboard representation and back is accomplished with the pair of functions f = ntz and g = lambda s: 1 << s. Calling these f and g respectively note that we have f(g(f(x)) == f(x), g(f(g(x))) == g(x)

popcnt (population count)

If we want to know the number of elements in the subset of {0,1,…,n**2-1} represented by a bitboard then we compute the so-called popcnt, which is the number of 1 bits in the binary representation. Again, there are bit hacks for this. A 64-bit popcnt algorithm written in Python is as follows:

def popcnt(x):  # 0 <= x < 2**64
    k1 = 0x5555555555555555
    k2 = 0x3333333333333333
    k4 = 0x0f0f0f0f0f0f0f0f
    kf = 0x0101010101010101
    x = x - ((x >> 1) & k1)
    x = (x & k2) + ((x >> 2) & k2)
    x = (x + (x >> 4)) & k4
    x = (x * kf) >> 56
    return x

Some modern processors have (will have?) popcnt and ntz instructions built-in (among others of the same flavor) for the machine word sizes.

Special Bitboards

Let’s distinguish some special bitboards that will appear. (Really, classes of bitboards for each n, but we take n=8 in our examples.)

The diagonal consists of the bits in the (n+1)*k positions for k in {0,...,(n-1)}.

def diagonal(n=8):
    return sum(2**((n+1)*k) for k in range(n))

print(diagonal())  # 9241421688590303745
html_bitboard(diagonal())

The anti-diagonal consists of the bits in the (n-1)*(k+1) positions for k in {0, ..., (n-1)}.

def antidiagonal(n=8):
    return sum(2**((n-1)*(k+1)) for k in range(n))

print(antidiagonal())  # 72624976668147840
html_bitboard(antidiagonal())

For an 8x8 bitboard, the eighth rank consists of the bits in positions 0, 1, …, 7, and so on to the first rank consisting of the last 8 positions. To generalize terminology, refer to the nth rank of an nxn bitboard as the end rank or top, and the first rank as the back rank or bottom.

For the 8x8 bitboard, the a-file consists of the bits in the positions 0, 8, 16, …, 56, the b-file of the bits in the positions 1, 9, 17, …, 57, and so on to the h-file. In the general case let’s define the left-file and the right-file.

Define columns to be files except with a differing indexing scheme: file a is column 0, file b is column 1, and so on.

Define rows to be ranks except with a different indexing scheme: rank 8 is row 0, rank 7 is row 1, and so on.

examples

Rank 3:

def rank(r, n=8):
    return (2**n-1) << (n*(n-r))

draw_html(rank(3))

Rank 8:

def top(n=8):
    return rank(n,n)

draw_html(top())

Rank-1:

def bottom(n=8):
    return rank(1,n)

draw_html(bottom())

The a-file:

def left(n=8):
    return sum(2**(n*k) for k in range(n))

draw_html(left())

The h-file:

def right(n=8):
    return sum(2**(n*(k+1)-1) for k in range(n))

draw_html(right())

Using lookup tables

One of the applications one might use a bitboard for in a chess engine is as a masking function: proj = lambda x: x & mask. If the mask only allows eight bits through (i.e. popcnt(mask)==8) then there are only 256 values in the range of proj. What we would like is an efficient algorithm for computing a perfect hash (i.e. a bijection) f from the range of proj to the integers 0,1,…255, and we would prefer if the properties f(x|y)=f(x)|f(y), f(x&y)=f(x)&f(y), and f(~x)=~f(x) all held, i.e. that f was an isomorphism of Boolean algebras. We’d also like if f preserved order (or perhaps reversed it).

The approach I’ve cooked up here does not seem to be directly in Henry Warren’s Hacker’s Delight book, but it is inspired by the recurring trick (appearing in both ntz and popcnt above) of multiplying by “magic” numbers and rolling off all but the highest bit.

The idea is to look for functions of the form

lambda x: (magic*x)%(2**64) >> 56

Or, more generally, look for functions that can be built by combining bit shift operations and modular multiplications in $\mathbb{Z}_{2**64}$ in any order.

def modmul(x, y, m=(2**64)):
  return (x*y)%m

Mapping the bottom to the top:

f = lambda x, n: x >> (n**2-n)

Mapping the left to the bottom:

f = lambda x, n: modmul(x, antidiagonal(n), 2**(n**2))

Mapping the left to the bottom in reverse:

f = lambda x, n: modmul(x, diagonal(n), 2**(n**2))

It becomes clear that we can map the left to the bottom in any permutation we’d like by using 8x8 permutation matrices encoded as bitboards.

Mapping to the bottom:

f = lambda x, n: modmul(x, left(n), 2**(n**2))

We show an example mapping the bits of the diagonal to the bottom.

We show an example mapping the bits of the antidiagonal to the bottom.

Exercise. Show this works not just for diagonals but for any permutation matrix.

A full bit 8x8 bit matrix transposition algorithm is given in Section 7.3 of Hacker’s Delight:

y = (x & 0x8040201008040201)        |
    (x & 0x0080402010080402) << 7   |
    (x & 0x0000804020100804) << 14  |
    (x & 0x0000008040201008) << 21  |
    (x & 0x0000000080402010) << 28  |
    (x & 0x0000000000804020) << 35  |
    (x & 0x0000000000008040) << 42  |
    (x & 0x0000000000000080) << 49  |
    (x >>  7) & 0x0080402010080402  |
    (x >> 14) & 0x0000804020100804  |
    (x >> 21) & 0x0000008040201008  |
    (x >> 28) & 0x0000000080402010  |
    (x >> 35) & 0x0000000000804020  |
    (x >> 42) & 0x0000000000008040  |
    (x >> 49) & 0x0000000000000080

We note that 0x8040201008040201UL is our friend diagonal(8) from above.

Language Models Can Disassemble Code

2022-04-02T00:00:00-04:00

OpenAI Codex seems to have some understanding of assembly language. It can even disassemble.

Assembly

I prompted OpenAI Codex to get it to invent a simple test C program and gave instructions on compiling it to assembler code, and then had it autocomplete what it believed the result would be. Here are the results.

You can use the -S option to gcc to produce assembly code.

gcc -S source.c

This will produce a file called source.s.

For example, suppose that source.c contains the following code:

#include <stdio.h>

int main(void)
{
    int a = 1;
    int b = 2;

    printf("%d\n", a + b);
    return 0;
}

If you compile this with gcc -S source.c, you will get the following assembly code in the file source.s:

    .file   "source.c"
    .text
    .globl  main
    .type   main, @function
main:
.LFB0:
    .cfi_startproc
    pushq   %rbp
    .cfi_def_cfa_offset 16
    .cfi_offset 6, -16
    movq    %rsp, %rbp
    .cfi_def_cfa_register 6
    movl    $1, -4(%rbp)
    movl    $2, -8(%rbp)
    movl    -4(%rbp), %eax
    addl    -8(%rbp), %eax
    movl    %eax, %esi
    movl    $.LC0, %edi
    movl    $0, %eax
    call    printf
    movl    $0, %eax
    popq    %rbp
    .cfi_def_cfa 7, 8
    ret
    .cfi_endproc
.LFE0:
    .size   main, .-main
    .ident  "GCC: (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609"
    .section    .note.GNU-stack,"",@progbits

Let’s explain roughly how this works, and some of the confusing notation and labels. Note: I wrote this line, and then it autocompleted the following explanation:

First, we see that the main function is defined:

    .globl  main
    .type   main, @function
main:

The .globl directive tells the assembler that the main symbol is global, i.e. that it can be accessed from other files.

The .type directive tells the assembler that the symbol main is a function.

The main: label tells the assembler that the next few lines of code are the definition of the main function.

Next, we see the following:

.LFB0:
    .cfi_startproc
    pushq   %rbp
    .cfi_def_cfa_offset 16
    .cfi_offset 6, -16
    movq    %rsp, %rbp
    .cfi_def_cfa_register 6

This is the function prologue.

The function prologue is code that is executed before the function body.

The .LFB0: label is a local label that is used to mark the beginning of the function prologue.

The .cfi_startproc directive tells the assembler that the next few lines are the function prologue.

The pushq %rbp instruction pushes the value of the rbp register onto the stack.

The .cfi_def_cfa_offset 16 directive tells the assembler that the stack pointer is 16 bytes away from the current frame pointer.

The .cfi_offset 6, -16 directive tells the assembler that the current frame pointer is 16 bytes away from the previous frame pointer.

The movq %rsp, %rbp instruction moves the value of the stack pointer into the frame pointer.

The .cfi_def_cfa_register 6 directive tells the assembler that the current frame pointer is rbp.

Next, we see the following:

    movl    $1, -4(%rbp)
    movl    $2, -8(%rbp)

These instructions move the values 1 and 2 into the variables a and b respectively.

Next, we see the following:

    movl    -4(%rbp), %eax
    addl    -8(%rbp), %eax
    movl    %eax, %esi
    movl    $.LC0, %edi
    movl    $0, %eax
    call    printf

These instructions move the value of a into the eax register, add the value of b to the eax register, move the result into the esi register, move the address of the string "%d\n" into the edi register, move the value 0 into the eax register, and call the printf function.

Next, we see the following:

    movl    $0, %eax
    popq    %rbp
    .cfi_def_cfa 7, 8
    ret
    .cfi_endproc

This is the function epilogue.

The function epilogue is code that is executed after the function body.

The movl $0, %eax instruction moves the value 0 into the eax register.

The popq %rbp instruction pops the value of the rbp register off of the stack.

The .cfi_def_cfa 7, 8 directive tells the assembler that the stack pointer is 8 bytes away from the current frame pointer.

The ret instruction returns from the function, and the .cfi_endproc directive tells the assembler that the function epilogue ends here.

Finally, we see the following:

.LFE0:
    .size   main, .-main

The .LFE0: label is a local label that is used to mark the end of the function.

The .size directive tells the assembler that the size of the main function is the difference between the main label and the .LFE0 label.

Dissassembly

Human writing.

Let’s talk about disassembly. For this one, I wrote (by hand) the following source.c file:

#include <stdio.h>

int main(void)
{
    int a = 1;
    int b = 1;
    for (int i = 0; i < 10; ++ i) {
      int c = a + b;
      a = b;
      b = c;
    }
    printf("%d\n", a + b);
    return 0;
}

We compile it to assembly with gcc -S source.c, yielding source.s:

	.file	"source.c"
	.text
	.section	.rodata
.LC0:
	.string	"%d\n"
	.text
	.globl	main
	.type	main, @function
main:
.LFB0:
	.cfi_startproc
	endbr64
	pushq	%rbp
	.cfi_def_cfa_offset 16
	.cfi_offset 6, -16
	movq	%rsp, %rbp
	.cfi_def_cfa_register 6
	subq	$16, %rsp
	movl	$1, -16(%rbp)
	movl	$1, -12(%rbp)
	movl	$0, -8(%rbp)
	jmp	.L2
.L3:
	movl	-16(%rbp), %edx
	movl	-12(%rbp), %eax
	addl	%edx, %eax
	movl	%eax, -4(%rbp)
	movl	-12(%rbp), %eax
	movl	%eax, -16(%rbp)
	movl	-4(%rbp), %eax
	movl	%eax, -12(%rbp)
	addl	$1, -8(%rbp)
.L2:
	cmpl	$9, -8(%rbp)
	jle	.L3
	movl	-16(%rbp), %edx
	movl	-12(%rbp), %eax
	addl	%edx, %eax
	movl	%eax, %esi
	leaq	.LC0(%rip), %rdi
	movl	$0, %eax
	call	printf@PLT
	movl	$0, %eax
	leave
	.cfi_def_cfa 7, 8
	ret
	.cfi_endproc
.LFE0:
	.size	main, .-main
	.ident	"GCC: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0"
	.section	.note.GNU-stack,"",@progbits
	.section	.note.gnu.property,"a"
	.align 8
	.long	 1f - 0f
	.long	 4f - 1f
	.long	 5
0:
	.string	 "GNU"
1:
	.align 8
	.long	 0xc0000002
	.long	 3f - 2f
2:
	.long	 0x3
3:
	.align 8
4:

Note: to clarify, the above listing was from gcc, not OpenAI codex. We are going to have it try to reverse-engineer it. This is called disassembly. To this end I prompted the OpenAI codex with the compilation instruction, the above assembly code listing, and the next two paragraphs, and you can see the results:

By inspecting this and understanding what is happening, we can reconstruct a possible C program source.c. This is called disassembly.

Here is one possible value for the file source.c:

#include <stdio.h>

int main() {
    int a = 1;
    int b = 1;
    for (int i = 0; i < 9; i++) {
        int temp = a;
        a = b;
        b = temp + b;
    }
    printf("%d\n", a + b);
}

Looks like it made an off-by-one error. Stupid computer!

A Free Man’s Worship

2022-01-23T00:00:00-05:00

A Free Man’s Worship, by Bertrand Russell

To Dr. Faustus in his study Mephistopheles told the history of the Creation, saying:

“The endless praises of the choirs of angels had begun to grow wearisome; for, after all, did he not deserve their praise? Had he not given them endless joy? Would it not be more amusing to obtain undeserved praise, to be worshipped by beings whom he tortured? He smiled inwardly, and resolved that the great drama should be performed.

“For countless ages the hot nebula whirled aimlessly through space. At length it began to take shape, the central mass threw off planets, the planets cooled, boiling seas and burning mountains heaved and tossed, from black masses of cloud hot sheets of rain deluged the barely solid crust. And now the first germ of life grew in the depths of the ocean, and developed rapidly in the fructifying warmth into vast forest trees, huge ferns springing from the damp mould, sea monsters breeding, fighting, devouring, and passing away. And from the monsters, as the play unfolded itself, Man was born, with the power of thought, the knowledge of good and evil, and the cruel thirst for worship. And Man saw that all is passing in this mad, monstrous world, that all is struggling to snatch, at any cost, a few brief moments of life before Death’s inexorable decree. And Man said: ‘There is a hidden purpose, could we but fathom it, and the purpose is good; for we must reverence something, and in the visible world there is nothing worthy of reverence.’ And Man stood aside from the struggle, resolving that God intended harmony to come out of chaos by human efforts. And when he followed the instincts which God had transmitted to him from his ancestry of beasts of prey, he called it Sin, and asked God to forgive him. But he doubted whether he could be justly forgiven, until he invented a divine Plan by which God’s wrath was to have been appeased. And seeing the present was bad, he made it yet worse, that thereby the future might be better. And he gave God thanks for the strength that enabled him to forgo even the joys that were possible. And God smiled; and when he saw that Man had become perfect in renunciation and worship, he sent another sun through the sky, which crashed into Man’s sun; and all returned again to nebula.

“‘Yes,’ he murmured, ‘it was a good play; I will have it performed again.’”

Such, in outline, but even more purposeless, more void of meaning, is the world which Science presents for our belief. Amid such a world, if anywhere, our ideals henceforward must find a home. That Man is the product of causes which had no prevision of the end they were achieving; that his origin, his growth, his hopes and fears, his loves and his beliefs, are but the outcome of accidental collocations of atoms; that no fire, no heroism, no intensity of thought and feeling, can preserve an individual life beyond the grave; that all the labours of the ages, all the devotion, all the inspiration, all the noonday brightness of human genius, are destined to extinction in the vast death of the solar system, and that the whole temple of Man’s achievement must inevitably be buried beneath the debris of a universe in ruins–all these things, if not quite beyond dispute, are yet so nearly certain, that no philosophy which rejects them can hope to stand. Only within the scaffolding of these truths, only on the firm foundation of unyielding despair, can the soul’s habitation henceforth be safely built.

How, in such an alien and inhuman world, can so powerless a creature as Man preserve his aspirations untarnished? A strange mystery it is that Nature, omnipotent but blind, in the revolutions of her secular hurryings through the abysses of space, has brought forth at last a child, subject still to her power, but gifted with sight, with knowledge of good and evil, with the capacity of judging all the works of his unthinking Mother. In spite of Death, the mark and seal of the parental control, Man is yet free, during his brief years, to examine, to criticise, to know, and in imagination to create. To him alone, in the world with which he is acquainted, this freedom belongs; and in this lies his superiority to the resistless forces that control his outward life.

The savage, like ourselves, feels the oppression of his impotence before the powers of Nature; but having in himself nothing that he respects more than Power, he is willing to prostrate himself before his gods, without inquiring whether they are worthy of his worship. Pathetic and very terrible is the long history of cruelty and torture, of degradation and human sacrifice, endured in the hope of placating the jealous gods: surely, the trembling believer thinks, when what is most precious has been freely given, their lust for blood must be appeased, and more will not be required. The religion of Moloch–as such creeds may be generically called–is in essence the cringing submission of the slave, who dare not, even in his heart, allow the thought that his master deserves no adulation. Since the independence of ideals is not yet acknowledged, Power may be freely worshipped, and receive an unlimited respect, despite its wanton infliction of pain.

But gradually, as morality grows bolder, the claim of the ideal world begins to be felt; and worship, if it is not to cease, must be given to gods of another kind than those created by the savage. Some, though they feel the demands of the ideal, will still consciously reject them, still urging that naked Power is worthy of worship. Such is the attitude inculcated in God’s answer to Job out of the whirlwind: the divine power and knowledge are paraded, but of the divine goodness there is no hint. Such also is the attitude of those who, in our own day, base their morality upon the struggle for survival, maintaining that the survivors are necessarily the fittest. But others, not content with an answer so repugnant to the moral sense, will adopt the position which we have become accustomed to regard as specially religious, maintaining that, in some hidden manner, the world of fact is really harmonious with the world of ideals. Thus Man creates God, all-powerful and all-good, the mystic unity of what is and what should be.

But the world of fact, after all, is not good; and, in submitting our judgment to it, there is an element of slavishness from which our thoughts must be purged. For in all things it is well to exalt the dignity of Man, by freeing him as far as possible from the tyranny of non-human Power. When we have realised that Power is largely bad, that man, with his knowledge of good and evil, is but a helpless atom in a world which has no such knowledge, the choice is again presented to us: Shall we worship Force, or shall we worship Goodness? Shall our God exist and be evil, or shall he be recognised as the creation of our own conscience?

The answer to this question is very momentous, and affects profoundly our whole morality. The worship of Force, to which Carlyle and Nietzsche and the creed of Militarism have accustomed us, is the result of failure to maintain our own ideals against a hostile universe: it is itself a prostrate submission to evil, a sacrifice of our best to Moloch. If strength indeed is to be respected, let us respect rather the strength of those who refuse that false “recognition of facts” which fails to recognise that facts are often bad. Let us admit that, in the world we know, there are many things that would be better otherwise, and that the ideals to which we do and must adhere are not realised in the realm of matter. Let us preserve our respect for truth, for beauty, for the ideal of perfection which life does not permit us to attain, though none of these things meet with the approval of the unconscious universe. If Power is bad, as it seems to be, let us reject it from our hearts. In this lies Man’s true freedom: in determination to worship only the God created by our own love of the good, to respect only the heaven which inspires the insight of our best moments. In action, in desire, we must submit perpetually to the tyranny of outside forces; but in thought, in aspiration, we are free, free from our fellow-men, free from the petty planet on which our bodies impotently crawl, free even, while we live, from the tyranny of death. Let us learn, then, that energy of faith which enables us to live constantly in the vision of the good; and let us descend, in action, into the world of fact, with that vision always before us.

When first the opposition of fact and ideal grows fully visible, a spirit of fiery revolt, of fierce hatred of the gods, seems necessary to the assertion of freedom. To defy with Promethean constancy a hostile universe, to keep its evil always in view, always actively hated, to refuse no pain that the malice of Power can invent, appears to be the duty of all who will not bow before the inevitable. But indignation is still a bondage, for it compels our thoughts to be occupied with an evil world; and in the fierceness of desire from which rebellion springs there is a kind of self-assertion which it is necessary for the wise to overcome. Indignation is a submission of our thoughts, but not of our desires; the Stoic freedom in which wisdom consists is found in the submission of our desires, but not of our thoughts. From the submission of our desires springs the virtue of resignation; from the freedom of our thoughts springs the whole world of art and philosophy, and the vision of beauty by which, at last, we half reconquer the reluctant world. But the vision of beauty is possible only to unfettered contemplation, to thoughts not weighted by the load of eager wishes; and thus Freedom comes only to those who no longer ask of life that it shall yield them any of those personal goods that are subject to the mutations of Time.

Although the necessity of renunciation is evidence of the existence of evil, yet Christianity, in preaching it, has shown a wisdom exceeding that of the Promethean philosophy of rebellion. It must be admitted that, of the things we desire, some, though they prove impossible, are yet real goods; others, however, as ardently longed for, do not form part of a fully purified ideal. The belief that what must be renounced is bad, though sometimes false, is far less often false than untamed passion supposes; and the creed of religion, by providing a reason for proving that it is never false, has been the means of purifying our hopes by the discovery of many austere truths.

But there is in resignation a further good element: even real goods, when they are unattainable, ought not to be fretfully desired. To every man comes, sooner or later, the great renunciation. For the young, there is nothing unattainable; a good thing desired with the whole force of a passionate will, and yet impossible, is to them not credible. Yet, by death, by illness, by poverty, or by the voice of duty, we must learn, each one of us, that the world was not made for us, and that, however beautiful may be the things we crave, Fate may nevertheless forbid them. It is the part of courage, when misfortune comes, to bear without repining the ruin of our hopes, to turn away our thoughts from vain regrets. This degree of submission to Power is not only just and right: it is the very gate of wisdom.

But passive renunciation is not the whole of wisdom; for not by renunciation alone can we build a temple for the worship of our own ideals. Haunting foreshadowings of the temple appear in the realm of imagination, in music, in architecture, in the untroubled kingdom of reason, and in the golden sunset magic of lyrics, where beauty shines and glows, remote from the touch of sorrow, remote from the fear of change, remote from the failures and disenchantments of the world of fact. In the contemplation of these things the vision of heaven will shape itself in our hearts, giving at once a touchstone to judge the world about us, and an inspiration by which to fashion to our needs whatever is not incapable of serving as a stone in the sacred temple.

Except for those rare spirits that are born without sin, there is a cavern of darkness to be traversed before that temple can be entered. The gate of the cavern is despair, and its floor is paved with the gravestones of abandoned hopes. There Self must die; there the eagerness, the greed of untamed desire must be slain, for only so can the soul be freed from the empire of Fate. But out of the cavern the Gate of Renunciation leads again to the daylight of wisdom, by whose radiance a new insight, a new joy, a new tenderness, shine forth to gladden the pilgrim’s heart.

When, without the bitterness of impotent rebellion, we have learnt both to resign ourselves to the outward rules of Fate and to recognise that the non-human world is unworthy of our worship, it becomes possible at last so to transform and refashion the unconscious universe, so to transmute it in the crucible of imagination, that a new image of shining gold replaces the old idol of clay. In all the multiform facts of the world–in the visual shapes of trees and mountains and clouds, in the events of the life of man, even in the very omnipotence of Death–the insight of creative idealism can find the reflection of a beauty which its own thoughts first made. In this way mind asserts its subtle mastery over the thoughtless forces of Nature. The more evil the material with which it deals, the more thwarting to untrained desire, the greater is its achievement in inducing the reluctant rock to yield up its hidden treasures, the prouder its victory in compelling the opposing forces to swell the pageant of its triumph. Of all the arts, Tragedy is the proudest, the most triumphant; for it builds its shining citadel in the very centre of the enemy’s country, on the very summit of his highest mountain; from its impregnable watchtowers, his camps and arsenals, his columns and forts, are all revealed; within its walls the free life continues, while the legions of Death and Pain and Despair, and all the servile captains of tyrant Fate, afford the burghers of that dauntless city new spectacles of beauty. Happy those sacred ramparts, thrice happy the dwellers on that all-seeing eminence. Honour to those brave warriors who, through countless ages of warfare, have preserved for us the priceless heritage of liberty, and have kept undefiled by sacrilegious invaders the home of the unsubdued.

But the beauty of Tragedy does but make visible a quality which, in more or less obvious shapes, is present always and everywhere in life. In the spectacle of Death, in the endurance of intolerable pain, and in the irrevocableness of a vanished past, there is a sacredness, an overpowering awe, a feeling of the vastness, the depth, the inexhaustible mystery of existence, in which, as by some strange marriage of pain, the sufferer is bound to the world by bonds of sorrow. In these moments of insight, we lose all eagerness of temporary desire, all struggling and striving for petty ends, all care for the little trivial things that, to a superficial view, make up the common life of day by day; we see, surrounding the narrow raft illumined by the flickering light of human comradeship, the dark ocean on whose rolling waves we toss for a brief hour; from the great night without, a chill blast breaks in upon our refuge; all the loneliness of humanity amid hostile forces is concentrated upon the individual soul, which must struggle alone, with what of courage it can command, against the whole weight of a universe that cares nothing for its hopes and fears. Victory, in this struggle with the powers of darkness, is the true baptism into the glorious company of heroes, the true initiation into the overmastering beauty of human existence. From that awful encounter of the soul with the outer world, enunciation, wisdom, and charity are born; and with their birth a new life begins. To take into the inmost shrine of the soul the irresistible forces whose puppets we seem to be–Death and change, the irrevocableness of the past, and the powerlessness of Man before the blind hurry of the universe from vanity to vanity–to feel these things and know them is to conquer them.

This is the reason why the Past has such magical power. The beauty of its motionless and silent pictures is like the enchanted purity of late autumn, when the leaves, though one breath would make them fall, still glow against the sky in golden glory. The Past does not change or strive; like Duncan, after life’s fitful fever it sleeps well; what was eager and grasping, what was petty and transitory, has faded away, the things that were beautiful and eternal shine out of it like stars in the night. Its beauty, to a soul not worthy of it, is unendurable; but to a soul which has conquered Fate it is the key of religion.

The life of Man, viewed outwardly, is but a small thing in comparison with the forces of Nature. The slave is doomed to worship Time and Fate and Death, because they are greater than anything he finds in himself, and because all his thoughts are of things which they devour. But, great as they are, to think of them greatly, to feel their passionless splendour, is greater still. And such thought makes us free men; we no longer bow before the inevitable in Oriental subjection, but we absorb it, and make it a part of ourselves. To abandon the struggle for private happiness, to expel all eagerness of temporary desire, to burn with passion for eternal things–this is emancipation, and this is the free man’s worship. And this liberation is effected by a contemplation of Fate; for Fate itself is subdued by the mind which leaves nothing to be purged by the purifying fire of Time.

United with his fellow-men by the strongest of all ties, the tie of a common doom, the free man finds that a new vision is with him always, shedding over every daily task the light of love. The life of Man is a long march through the night, surrounded by invisible foes, tortured by weariness and pain, towards a goal that few can hope to reach, and where none may tarry long. One by one, as they march, our comrades vanish from our sight, seized by the silent orders of omnipotent Death. Very brief is the time in which we can help them, in which their happiness or misery is decided. Be it ours to shed sunshine on their path, to lighten their sorrows by the balm of sympathy, to give them the pure joy of a never-tiring affection, to strengthen failing courage, to instil faith in hours of despair. Let us not weigh in grudging scales their merits and demerits, but let us think only of their need–of the sorrows, the difficulties, perhaps the blindnesses, that make the misery of their lives; let us remember that they are fellow-sufferers in the same darkness, actors in the same tragedy as ourselves. And so, when their day is over, when their good and their evil have become eternal by the immortality of the past, be it ours to feel that, where they suffered, where they failed, no deed of ours was the cause; but wherever a spark of the divine fire kindled in their hearts, we were ready with encouragement, with sympathy, with brave words in which high courage glowed.

Brief and powerless is Man’s life; on him and all his race the slow, sure doom falls pitiless and dark. Blind to good and evil, reckless of destruction, omnipotent matter rolls on its relentless way; for Man, condemned to-day to lose his dearest, to-morrow himself to pass through the gate of darkness, it remains only to cherish, ere yet the blow falls, the lofty thoughts that ennoble his little day; disdaining the coward terrors of the slave of Fate, to worship at the shrine that his own hands have built; undismayed by the empire of chance, to preserve a mind free from the wanton tyranny that rules his outward life; proudly defiant of the irresistible forces that tolerate, for a moment, his knowledge and his condemnation, to sustain alone, a weary but unyielding Atlas, the world that his own ideals have fashioned despite the trampling march of unconscious power.

Shaun Harker

Chess Engine Bit Hacks

Bitboards

Visualizing Bitboards

Bitboards as subsets of {0,1,...,n**2-1}

logic

ntz (number of trailing zeros)

popcnt (population count)

Special Bitboards

examples

Using lookup tables

Mapping the bottom to the top:

Mapping the left to the bottom:

Mapping the left to the bottom in reverse:

Mapping to the bottom:

Related

Language Models Can Disassemble Code

Assembly

Dissassembly

A Free Man’s Worship

A Free Man’s Worship, by Bertrand Russell

Bitboards as subsets of `{0,1,...,n**2-1}`