Happy Easter

My hometown decided to have some Easter decoration this year, sponsored by the city council. Gets you into the spirit, doesn't it ?

Here's some more:

Updated Draft of Paper on Newcomblike Problems

Abstract:

This article discusses the family of Newcomblike problems in the context of reinforcement learning. This reframes the problem of rational decision making as one of obtaining maximal rewards in a wide range of environments. Newcomblike problems are characterized by correlations between agent and environment policies. Such correlations are likely if the environment contains other agents with similar architectures, which is a realistic assumption in practice. An optimal policy, taking into account these correlations, is given for known environments. 

Download the .pdf here.

Enjoy.

Back Again

Happy new year!

Big Bad Uzbek Solar Laser

I'm ending this year with an uplifting example of engineering, Uzbekistan's 1MW solar-powered Nd:YAG laser research facility. I'm not exactly sure what the intended research objective is, but if I had to wager a guess, I'd say it has something to do with energy transmission from orbit. What (relatively) poor, resource-rich, Uzbekistan, with its large deserts, is to gain from orbital solar power is somewhat unclear, so I may be wrong here. Still, while I'm posting this for the obvious Dr. Scaramanga connotation, this project is an example of rationally justifiable, cost-efficient science with a clear objective, at least if compared to madman's schemes such as magnetic confinement fusion, the Space Launch System, or the occasional FP7 project (link in German), or most of what I've seen at AGI-11 this year. Cheers to 2012 !

A Change to the Link List was Overdue

I'm no longer linking to the Singularity institute blog, as I do no longer wish to be associated with this organization in any way. Dresden Codak gets the boot for displaying Megatokyo Syndrome (glacial pacing, asking for donations for broken computers, etc.) and Pink Tentacle seems, sadly, to be defunct since April (I dearly hope this guy is OK).

Replacing them are Gene Expression, a blog on "human evolution, genetics, genomics and their interstices"; something like a 21st century Rassenlehre without any of the associated wickedness; Hark! A Vagrant, a webcomic, by the wonderful Kate Beaton, on topics like Sherlock Holmes, Canadians, Tesla, Chopin and Liszt, book covers, and much more; and Monsieur LaMoe (moderately NSFW), who lends a unique voice to the world's hikikomori, and whose status as a real person vs. a consortium of writers is still under debate. (I'm leaning somewhat towards "real person".)

Loriot is Dead

German comedian Loriot died yesterday, aged 87. He will be missed.

Paper Drafts

Here are draft versions of two short papers. They are on machine ethics and decision theory in the context of reinforcement learning. Comments are welcome.

Altruistic Reinforcement Learning

Abstract:

This article presents a modification of reinforcement learning where an agent’s action lead to rewards being received by a second agent interacting with same environment. This model can be useful in the development of powerful AIs. Agent policies are proposed for dealing with observable rewards, with non-observable rewards in perfectly rational agents, and with non-observable rewards in bounded rational agents.

Newcomblike Problems and Optimal Agents

Abstract:

This article discusses the family of Newcomblike problems in the context of reinforcement learning. It reframes the problem of rational decision making as one of obtaining maximal rewards in a wide range of environments. Newcomblike problems are characterized by correlations between agent and environment policies. An optimal policy, taking into account these correlations, is given for known environments. For unknown environments, a quality criterion for policies is formulated.

Spy

The picture above shows an East German spy. I am not making this up.

It was taken during an official course on how spies should dress, and was recently dug out by artist Simon Menner. See his online Stasi gallery for more spy pictures.

Make now mistake - despite the occasional screw-up, the East German state security was in fact a (deadly) effective, and ruthless, organization.

Another spy pic, more Austin Powers style:

The End of the Shuttle Program

LUKE:You were raised Jewish, right?

ELIEZER: Well that’s what I used to think, and then at one point I was watching a space shuttle launch on TV and getting tears in my eyes and realizing that I didn’t really get tears in my eyes for anything Judaism-related. That was when I realized that my childhood religion that I’d sort of grown away from over time, but still had the power to bring tears to my eyes, wasn’t Judaism so much as space travel.

(Luke Muehlhauser interviewing Eliezer Yudkowsky)

Celebrating the last launch of a space shuttle earlier today, a little music video by Yours Truly. Yoko Kanno's "BLUE" (performed by Mai Yamane, Yoko Kanno & the Seatbelts) from Cowboy Bebop set to Discovery's last launch, and a slideshow of a few shuttle-related images. This is in personal memoriam of A.K., who didn't make it for Discovery's final flight by a few weeks.

The 34 Year Old Scientist

Watson discovered the structure of DNA when he was 24. Heisenberg formulated the uncertainty principle when he was 25. Newton claimed to have developed his gravitational theory when he was 24. Darwin embarked on the Beagle when he was 22.

I could go on for hours like this.

But wait. Crick was 37 when he discovered the structure of DNA. Schroedinger was 38 when he published on wave mechanics. Newton cast his gravitational theory in mathematical form when he was 37, his earlier insights likely being purely speculative. Darwin finalized the theory of selection when he was 47 (Wallace being 33 at the time.)

Of course I may be biased here. Ever so many examples and counterexamples don't prove a point. One has to look at the data that's out there.

Falagas et al. (2008) ask "At what age do biomedical scientists do their best work?" and answer with the following age histogram of the top 5 highly cited articles for a random subset of 300 bioscientists:

(The corresponding histogram for the single-most cited paper looks noisier, propably based on people's tendency to cite summary reviews written in later years.)

Costas et al. (2010) perform a more thorough analysis for scientists working at the Spanish National Research Council. Their results indicate that while the number of publications per scientist per year increases somewhat with age, the expected number of citations per publication decreases. However, their study lumps together all age groups younger than forty. (Top, Low, and Medium refer to three performance classes of researchers).

Finally, the widely cited and awesomely titled Kanazawa (2003) "Why productivity fades with age: The crime–genius connection" examines the age at what 280 famous scientists made their single key contribution to science:

Quite depressing overall, but it seems you don't have to go fishing before you turn 40 (Einstein was 41 in 1920). At 34, chances are 50:50 that the best of your work still lies ahead. Even better (or worse, depending on your personal situation), the corresponding curve for the 72 scientists in Kanazawa's dataset who never married looks significantly different:

(This might be somewhat confounded by the unknown fraction of (closet) gay scientists in the sample.) Newton, Erdös, Tesla (and Anton Bruckner) immediately come to my mind as straight men who denied themselves the pleasures of female company, and were productive well into their forties, or later. (The catch being that Tesla and Newton became funny at around 50; Erdös was born that way; and Bruckner, well, that depends on your opinion on watertight underwear.) Crick, Schroedinger and Darwin were all married.

Falagas, M. E., Ierodiakonou, V. and Alexiou, V.G. (2008): At what age do biomedical scientists do their best work? The FASEB Journal 22,12 4067-4070

Costas, R, Van Leeuwen, T.N., Bordons, M (2010): A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact. J ournal of the American Society for Information Science and Technology 61, 8 1564–1581

Kanazawa, S. (2003): Why productivity fades with age: The crime-genius connection. Journal of Research in Personality, 37, 257–272

Credit Assignment & The Singularity

There is no limit to what a man can do so long as he does not care a straw who gets the credit for it.

(Charles Edward Montague)

Technological breakthroughs are often difficult to assign to a particular person, time, or place. Some methods are "in the air", and are reinvented independently in short succession. Financial and emotional motives complicate matters further. Inventors may regard each other's ideas as special cases of their own, more general insights. In 1910 the Smithsonian Institute congratulated the Wright brothers "for bringing [Samuel P. Langley's] aerodrome..to the commercial and practical stage", and only in 1942 issued an official statement that the Wrights, not Langley, had invented the airplane. Only in 1914, after 8 years of court struggles, and strong opposition from Glenn Curtiss, was the Wright's plane patent declared valid. I myself am currently witnessing, in my field of work, a similar patent war, albeit at a much smaller scale.

However, with the Singularity, things are different, as a friend recently pointed out to me over dinner in Berkeley. (Of course, the same idea had occurred to me already, independently!) If a breakthrough in AI results in the creation of a very powerful entity, this entity will likely find it trivial to sort out who contributed how much to its coming into existence. If the entity is benevolent, it will likely take care of proper credit assignment. Spin, PR, old-boy-networks and lawsuits are probably no match to superintelligence, nanotechnology, and non-invasive brain scanning.

With that in mind, the hopeful AI researcher can focus his attention on maximizing the chances for a benevolent Singularity. That means he should publish his work. Credit assignment can be of higher order, such as when his published ideas enable someone else's breakthrough. In the context of a normal invention, this is less desirable than holding back ideas and achieving the breakthrough himself, a little later maybe. In the context of the Singularity, however, earlier is better, all else equal, given ~60 million people dying per year. While his competitor may have beat him to the finishing line, he enabled the competitor's early success by releasing his ideas. That the competitor won't acknowledge this, is no longer a problem after the Singularity, and speeding things up by a mere week may save a million lives.

So if you have an idea that could matter for AI, and the Singularity, set it free. You know it wants to be.

What I, as a Transhumanist, Believe.

As a transhumanist, I believe a world without fear is possible.

As a transhumanist, I believe the world is very, very, very, big.

As a a transhumanist, I believe I can become something else, and still be myself.

As a transhumanist, I believe in the power of truth.

As a transhumanist, I believe in happy endings.

Quote of the Day

"Reality is a non-ergodic partially observable uncertain unknown environment in which acquiring experience can be expensive."

Marcus Hutter, Feature Markov Decision Processes

Quote of the Day

"You know what they say the modern version of Pascal’s Wager is? Sucking up to as many Transhumanists as possible, just in case one of them turns into God."

Julie from Crystal Nights by Greg Egan

Is Google Growing Linearly ?

Could make sense if (1) a fixed percentage of total ad spending is moved to google every year and (2) the amount not yet moved over is much smaller than the total amount that will be eventually moved. Not too unreasonable. Fit is minimal-relative-error (least squares). Extrapolation to 2015 gives 48-56 Bio USD in annual sales and 10% year-on-year sales growth, to which the overall growth in ad spending may contribute a few percent.

Happy Baby Bunny Pony

At Less Wrong, Alicorn (or is it really Eliezer?) discusses how the fact that some (pictures of) baby animals are more cute than all (pictures of) human babies fits with human evolutionary psychology. We're after all supposed to find our own offspring to be the cutest of all species. According to the theory of supernormal stimuli, proposed by Konrad Lorenz in the 1940s, this is due to the bunny possessing the features that make human babies cute, like big eyes, small nose, rounded forehead, to an even greater extent than any human baby does. This is certainly true, but why isn't our perception of such features maximized for values found in actual human babies ?

I think the answer lies in what isn't there in the bunny. If we ever encountered a real, living human baby with the eye/nose proportions and forehead curvature of the bunny pictured above, we'd be grossed out. This makes sense from an evolutionary point of view, (not from an ethical point of view), as the carrier of such body proportions would have no real chance of survival. The "cuteness ratios" are there, but our overall gestalt perception kicks in, and adds a big "gross!" factor. So mother nature needs not bother to make our ratio- or feature-based cuteness detection the shape of an inverted U, as low-level shape and texture perception will take care of any outliers. Within the region of "normal" babies, the bigger the eyes, the better. The bunny, however, is clearly not of human gestalt, is furry, has long ears, therefore doesn't trigger any "icky" response and can make our feature-based cuteness perception go berserk.

What's true for babies, also holds true for babes. Men find slender legs, big eyes, small chin, etc., attractive in women, but there's a limit that. Manga and anime, however, feature characters with extreme body proportions that still manage to be highly attractive to some people. While Scott McCloud proposes in his book Understanding Comics that the heavily abstracted visual style of cartoon characters serves primarily to allow a broad range of readers to recognize themselves in them, I would like to add that, by introducing a clear non-humanness, it also allows to explore regions of cuteness-feature-space that are off-limits to naturalistic art forms.

You can of course combine it all into ueber-cute cartoon animal babies.

As a transhumanist, all this offers me a nice glimpse of a future where we will have reeingineered our perception (and possibly our appearance) to accomodate an affective dynamic outside the human range.

Surreal Xmas Films

And we're back. Seems like in September, some wicked kids removed my batteries and left me outside in the rain to rust. Least that's what it feels. Maybe more on this later.

Over the years, I have, I don't know why, developed a cherished tradition of watching surreal films around Christmas. Maybe this is because it brings back memories of watching, as a kid, often somewhat surreal, at least from a kid's perspective, animated films on the 24th (think Animal Farm). Maybe it's because I feel the surrealism of Christmas itself (virgin, manger, kings, star, etc.) is completely lost on my surrounding.

A good point to start ist The Yellow Submarine (UK, 1968), (tvtropes). They showed this on Austrian public TV when I was in grade school (ca 1985), but for me, it got "pinned" to the Holidays in 1991, when I watched it on the afternoon of the 31st of Dec. in a hotel room in Hamburg, where I was with my mom to celebrate New Year's Eve with a harbor cruise (Austrians...). The Yellow Submarine features good music, superb animation, and, unique among the films mentioned here, a plotline more coherent than the average dream.

Fast forward to Christmas 2003, which is where I watched Angel's Egg (Japan, 1985), (tvtropes). I had the strangest feeling then, that I had already seen this, as a kid, around the time it was first released, which is highly unlikely for too many reasons to name. I really like re-watching this one almost every year.

But I will never again watch Jan Svankmajer's adaptation of Alice in Wonderland, (Czechoslovakia, 1988), which I rented from Blockbusters around Christmas 2005, when I was in Palo Alto. Set in a filthy, claustrophobic, apartment building in the Eastern Bloc, using stop-motion to achieve the creepiest imaginable effect (putting glass eyes in animal skulls, and animating them as disembodied skulls), this film put me in a mild shock-like state for days. You can find some scenes on youtube, but be warned.

Compared to this, Die Reise ins Glueck ("Journey into Bliss", Germany, 2004) by German dilettante director Wenzel Storch, is only mildly disturbing. Eight years in the making, the film features fantastic backdrops and stage props (most of them scrounged, or outright stolen), terrible acting, and a scene showing the mating of a sentient snail-shaped ship and a church, which, of course, results in the formation of a time-machine. I gave this, as a present, to a friend of mine, this Christmas. I think I'll opt for a tie next year.

The most surreal of all films in my list, however, is the Star Wars Holiday Special (USA, 1978) (tvtropes). I tried watching it this year, but, but ... Bea Arthur?!?!? This film is chock full of "Wait. I can't believe they did that." moments. And the advertisements fit seamlessly into the overall strangeness ("Tobor is robot spelled backwards"). If you've ever wanted to catch a glimpse of a parallel Universe, this is as good a substitute as you can probably get.

That Is So Amazingly Amazing I Think I'd Like To Steal It.

Makoto Shinkai's 5cm per Second is one of my favorite movies. Shinkai's storytelling, visuals, and sound effects show incredible attention to detail, and have made him the new top dog amongst Japanese animators.

Enter the Chinese Communist Party Propaganda Department, an their latest gift to mankind, "Soul's Window" (心灵之窗), a “moral anime,” featuring, "themes of selfless dedication to party and state...[]... and the heroic actions of the People’s Liberation Army ." (Hat tip to Sankaku Complex!)

The attentive critic may find some resemblance between Soul's.. and 5cm..:

See the many more screenshots like these at Sankaku Complex. This may be the biggest thing since the Great Lion Theft.

You can watch what seems to be a trailer for Soul's Window here or here.

Or you can watch Shinkai's one-minute short "A Gathering of Cats", which is approximately a Zillion times better, here.

Eight Ways To Spot A Mediocre Scientist.

It's easy to spot a poor scientist. Lack of publications, inability to communicate, and blatant incompetence are hard to overlook for more than a few hours. Sorting the mediocre from the first-class, however, is much more difficult. Brilliance sometimes looks like biasedness, and vice versa. Sub-par people may still publish good papers through a combination of luck, the right working environment, the right supervisor, and good funding. Eloquence can mask shallowness (for a while.) On the other hand, smart guys may be too young for a strong track record (or may have been unlucky), may be deliberately soft-spoken, may use self-deprecating humour, or may agree to stupid statements out of politeness.

One would need to have excellent insight into the relevant field in order to recognize true outperformance. If you're teaming up with a collaborator across domains (e.g. a computer scientist teaming up with a linguist) you will lack this level of insight. (If not, why team up?) Getting stuck with a mediocre scientist as a collaborator, student, supervisor or colleague is guaranteed to cause frustration, a dent in your career, lots of extra work, and mild to severe psychological damage. Over the years, I learned, the hard way, that mediocre scientists (henceforth MS) share some traits among themselves, which can serve as a early warnings:

1. Monologue
The MS likes to elaborate. Droning on and on, the MS will not have time to listen. Make casual, but relevant, statements, and check later whether he remembers any of them. Look for email responses to simple questions of over four pages in length.

2. Quirky Categorization; Meta-Theory
The MS has usually developed idiosyncratic systems of categories for several domains. He regards these, as well as his general meta-theory of scientific insight, as absolutely essential in order to get any work done. Without using those categories, nothing makes sense. Anyone not using his "system" (i.e. everyone except himself) has no chance to make real intellectual progress. He will, however, freely share this insight with anyone (remotely) interested (cf. point 1).

3. BSing outside of his Speciality
When discussing topics outside of his field of expertise, (which he will be eager to do, cf. point 1), the MS will quickly talk some form or another of utter BS, like, e.g. US per capita GDP being hundred times bigger than global average. The MS may be aware, and admit, that he has only the most casual knowledge of the domain in question, but will nevertheless deliver his statements with utmost certainty (cf. next point).

4. Preaching, Teaching
The MS is sure of what he says. Absolutely. He has had this discussion a hundred times before, with people smarter than you, and none of those people were able to provide solid counterarguments to his position. Sure, some tried, some tried really hard, but after a few hours of discussion, or a few 5-page-long email exchanges, their arguments all melted away, and they admitted defeat. Or at least stopped responding. Check the archives. The MS is as sure of his access to privileged knowledge as a maths teacher in front of grade-schoolers, and it shows.

5. Misses Appointments
When working with you, the MS will be late, will, at first, rush, to make up for the time lost, but then slow down, and suggest having a coffee or two together. Then he suddenly realizes he needs to be elsewhere, and leaves early.

6. Belittles Textbooks, People, Publications, Institutions, Methods...
Nothing is good enough for the MS. He will give you an earful about the textbooks being confusing, too detailed or too superficial, everyone except himself lacking true insight (cf. point 2), publications being based on flawed methodologies, institutions funding the wrong kind of people, and so on and so forth. This is not the usual whining of the underprivileged - he himself may have good funding. It's more of an aesthetic complaint.

7. Doesn't Follow the Literature
When Caliph Umar the Great ordered the books of the Library of Alexandria to burned for heating bathing water, he is said to have stated that "they will either contradict the Koran, in which case they are heresy, or they will agree with it, in which case they are superfluous." (Wikipedia says this is a hoax.) The MS, however, has an analogous view of the literature in his field (the "Koran" being his own work, in particular his meta-theory, cf. point 2). Pick his brain, and you will find he's completely unaware of publications which could help him a lot in his efforts.

8. Would Prefer to do His Own Stuff
The MS is an independent spirit. He will not compromise. He will not team up. He demands flexibility. He will not precommit. He will not allow anyone to "interfere" with his work. He'll play it by ear. While all this can also happen with a truly great mind working in a destructive environment, in combination with the seven points above it's an indication that you have an MS in front of you.

Carving Up Reality.

In a wonderful choice of words, the Wall Street Journal yesterday mentioned in an article that Agilent Technologies Inc. "[...] makes machines to analyze DNA, chemicals, sound waves and other items [...]".