Sunday, August 7, 2022

Human Rights A Very Bad Idea

 Human Rights

A Very Bad Idea

Interview of Raymond Geuss by Lawrence Hamilton for

Theoria: A Journal of Social and Political Theory

RAYMOND GEUSS, Professor of Philosophy at the University of Cambridge, is

one of the world’s most distinguished political philosophers. His recent books

include Philosophy and Real Politics (Princeton), Outside Ethics (Princeton),

Public Goods, Private Goods (Princeton) and History and Illusion in Politics

(Cambridge). This interview took place in his house in Cambridge, U.K., on

Wednesday 13 March 2013.

Lawrence Hamilton: The ‘first question, Raymond, is the most basic, and

it’s in two parts. First, what are human rights? Second, in History and Illusion

in Politics (CUP 2001) you speak of human rights as a kind of ‘puffery’ or

‘white magic’. Why?

Raymond Geuss:Well, to start with, I think the notion of a right is correctly

described in a lot of the philosophical literature as the notion of an individual

trump that a person holds. The political process and various legal processes

operate in different ways and according to various principles: we have negotiations,

hearings, forms of arbitration; we have legal cases, we have parliamentary

debates, in which diverse proposals and different claims are discussed

and some decision is reached. In the ideal case perhaps even some reconciliation

of different views and claims is reached or even some kind of consensus.

The notion of a right is the notion of someone having a recognised power to

intervene, restrict discussion and break the political process down. I can stop

the consideration of any course of action by playing my trump card. You’re

deciding whether or not to build a highway through my property, and you talk

about the pros and the cons, the utility and the possible disadvantages, but, if

it is my land, I can play my trump card, which is my rights-based claim ‘it’s

my private property’. Similarly, if the government want to incarcerate me simply

on suspicion, I can appeal to my right not to be imprisoned without a

public trial. Before the recent War on Terror that was a ‘right’ people were

Theoria, Issue 135, Vol. 60, No. 2 (June 2013): 83-103

doi:10.3167/th.2013.6013505 • ISSN 0040-5817 (Print) • ISSN 1558-5816 (Online)

supposed to have, and by appeal to it they were supposed to be able to stop the

government from doing this kind of action. So the notion of a right is the

notion of a trump in that sense.

The term ‘human’ can mean two different things in discussion of rights. A

‘human right’ can either mean a right for humans, so a human right to housing

is a right humans have to shelter of a particular kind and quality, as opposed,

say, to a right that farm animals or domestic pets have to a particular kind of

shelter. In a second sense, a ‘human right’ can mean a right that is supposed to

arise out of, or be in some ways grounded in, our mere humanity. The idea is

that some rights people have depend on specific positive political decisions,

but others depend simply on the fact that people are human beings, not on any

particular decision by any political body. For instance, in some countries certain

members of the clergy of particular religious groups might have been

given the right to exemption from particular forms of taxation because the

political authorities there have decided for whatever reason that this is a good

idea. This means that the authorities in question have set up a mechanism to

monitor taxation, collect taxes from members of some groups, but not from

the clergy, resolve disputes about whether a given person is or is not entitled

to the exemption, etc. In other countries these exemptions might not hold

because the authorities in those countries have made a different decision. The

right to an exemption in the first case is a (mere) ‘positive’ right. Those who

believe in ‘human rights based on our humanity alone’ think that in addition

to such cases, there are also other cases in which people have ‘rights’ independently

of any specific political decision or the existence of any effective

mechanism of enforcement. So there are two different senses of a human right:

a right for people, in the way in which an animal right is a right for animals, or

a human right is a right that purportedly arises out of our humanity alone and

not from the political process.

When thinking about human rights it is really important to see whether

these human rights are thought of as positive legal rights or as natural or innate

rights. That is, are the rights that individual human beings have construed as

powers or warrants that are given to them by the political process and by the

juridical process, given to them in a particular political system as part of that

system, or are they construed as something outside the political system, that

are given to them by God or by nature or by their human nature? In the second

case, they would be something outside the political process which intervenes

in that process.

When I talk about rights as ‘white magic’ or ‘puffery’, what I primarily

mean is the second conception, that is the concept of human rights as natural

or innate or as given prior to any kind of political process. There my simple

idea is that this notion is incoherent. We can perfectly well make sense of your

having a right, if this means that the political process operates in such a way

that the political authorities will enforce a certain claim that you make. That is

a perfectly coherent conception but this conception makes sense only if you’ve

84 Raymond Geuss, interviewed by Lawrence Hamilton

got a political agency, which has granted the right, an agency that will interpret

cases and determine when the right has been violated, and an agency that will

enforce the right. So if you have all of those things in place then you have a

clear idea of a right, but in most of the interesting cases, people want to speak

of ‘human rights’ precisely in cases in which these conditions are assumed

not to hold. I have an (innate) ‘human right’ not to be tortured, although the

local political regime does not recognise or fully enforce that right. So a ‘natural

(or innate) human right’ or one based on our mere humanity is modelled

on the idea of a positive right, but it is applied in a context in which all the real

content of that idea is absent, where the context for using the concept of ‘right’

is not present. The content of a concept is given by the existence of an interpreting

and enforcing mechanism. To speak of innate human rights, however,

is to say you’ve got a right, but there is no such enforcement mechanism. That

is why I think there is something inherently difficult, unclear or even incoherent

about the notion of an innate or natural human right. I say it is puffery

because you could imagine that there was an enforcement mechanism, namely

suppose you thought that I had magical powers and could harm you simply by

thinking something. Suppose that everybody in my society thought that I was

a powerful magician. I could cause you to fall ill simply by thinking about you

in an especially malevolent way (and perhaps performing certain rites). I am,

however, not a magician, I can’t harm you by projecting mental rays at you –

but if everyone believed that I could, there would be something self-reinforcing

about that belief, because then the idea that if you touched my property

you would be in big trouble, would have some kind of basis. You at any rate

would live in a state of uncomfortable anticipation of something I might do to

you; if your neighbours came to know what you had done, they might begin to

avoid you, etc.

So, what would be the point of saying that I have a right? One thing could

be that if I have a right, then God will intervene, or the state will intervene, to

protect that right. In the first case, one would have an innate right; in the second

a positive right. Either of them would be a way of giving some grounding

to that idea of a right, but another idea would be that you think that I could and

would intervene if you touched my property or tried to assault me. If I actually

had the power to harm you (magically) when you violate my rights, the question

of whether or not the state intervened might become irrelevant. But, of

course, I don’t have that power – that is, in fact, the whole point of positing

rights. If I could magically take care of myself, I wouldn’t need them. If, however,

you do presuppose that the intervention in question is state-intervention

– or the real, effective intervention of some state-like organisation – then what

you are talking about is a positive, not a natural or innate right to something.

If you assume that God will intervene, then you can introduce a clear sense of

a natural or innate right. It is what he would intervene to protect. But without

God or magic – the self-reinforcing notion that if you touch my property you

will regret it – you can have no substantive idea of a natural right.

Human Rights: A Very Bad Idea 85

‘White magic’, then, is the idea that I can hurt you (or, of that matter that

God will hurt you) if you violate my rights. This idea will work as a foundation

of rights, provided (but only provided) everybody believes it. It can be selfreinforcing.

But even if everybody believes it, it’s still not true – I don’t have

that magic power and neither does God – and that is the important thing to

understand here: that in politics, partly, we want to see what works; because

what people believe is in itself very important. But in politics we also want to

distinguish between the government intervening to protect my property, and

my property being protected because everyone makes a mistake about my

mental powers. That’s the point I’m trying to make here. That clarity in politics

requires looking beyond simply what will work, and understanding the mechanism

by which various things work correctly. So, even if you all believe that

I’m a witch and I can protect myself and my property and therefore I have got

a right that I can enforce, the whole structure is a misconception because it is

based on a false assumption (about my magical powers) and so you’re never

going to be able to trust it in the way you can trust something that has a different

status (such as being true).

Lawrence Hamilton: Your answer links to one or two further questions. I’m

going to take them first, focusing in particular on your example of your being

a magician and what follows from that even if everyone believes in your supposed

magical and effective powers, etc. I’d like to push you a little harder on

that. And thereafter I’ll come back to some other, related concerns. So, white

magic works best when most members of the group believe in it, as you’ve

just said …

Raymond Geuss: It doesn’t work best, it only works at all if people hold the

beliefs! That’s the point. Think about enforcement. How can enforcement

work? There are forms of natural enforcement, for instance ‘natural enforcement’

of a prohibition on drinking too much alcohol. The prohibition on drinking

too much alcohol enforces itself to some extent: you drink too much you

get sick! Nature enforces that prohibition. You take too many drugs, you get

ill. That’s one kind of enforcement.

Another kind of enforcement is the one in which society enforces some

prohibition through a specified agency, for instance, the police. Another kind

of enforcement is a self-perpetuating illusory enforcement: you all think that

if you touch my property I will harm you magically. These are different kinds

of enforcement and one must see them as very different. The natural forms of

enforcement are unproblematic. I have no problem with them. The state forms

of enforcement are also in some ways unproblematic, although we might raise

questions whether the proper formatting of that is as the enforcement of rights,

but the enforcement through consensus, as in the example in which everyone

thinks I am a witch, even though if it looks like it’s a robust form, is not

always robust, because people can change their beliefs. It is particularly not

86 Raymond Geuss, interviewed by Lawrence Hamilton

going to be robust if it’s false. As theorists and as political actors we have to

see what works, but we also have to understand what’s real and what’s true.

Let me give a historical example. In the Late Medieval and Early Modern

periods most people in Europe believed in the Christian God, and they believed

he had specific ideas about how humans should behave, and would enforce

these ideas. So under those circumstances, appeal to (God-given, natural)

‘human rights’ might have some kind of real effectiveness. If belief was really

monolithic, no one would be willing simply to ignore violations of what were

thought to be the rights God gave humans. God might smite you in this life,

but even if he did not, he would get you after death. This, of course, did not

mean that no one ever did violate those rights, but it also does not mean they

had no effect. This structure gave politics a particular shape. If you were a

powerful monarch you might be able to find or hire some clerics to give ‘interpretations’

of what God commanded (or what you were doing politically) that

made it seem compatible with Divine Law, but you would be unlikely to try to

claim that you simply didn’t care at all about whether what you did was religiously

acceptable. Given the state of belief and the institutions that existed,

that would have been a short road to political suicide. Or perhaps you could

have got away with that, but only if you were very powerful and very clever

indeed. So under those circumstances ‘human rights’ had some basis, but their

basis was not in God (who doesn’t exist) or nature, but in the existing structure

of institutions and beliefs, and the distribution of power.

Lawrence Hamilton: In a sense that’s an answer to the second question.

White magic works best when most members of the group believe in it but not

on those who don’t. What do you mean by this? Isn’t there also the possibility

of simply extending this to human rights? What do you mean by this with

respect to human rights in particular? In other words, can I believe, can I

choose to believe, that human rights as they’re defined do not actually exist in

reality, such that they no longer ‘work on me’?

Raymond Geuss: They don’t, or at any rate they might not, ‘work on you’

because the international regime of human rights, even the regime set up by

the United Nations, has no appropriate teeth. Part of the basic motivation for

‘human rights’ is that they are supposed to be outside or beyond politics; they

are something to which you can appeal when politics goes wrong. However,

what actually happens in the international regime of human rights even as it is

defined by the United Nations is that the powerful countries use claims about

rights to further their interests. They act to enforce rights when that is in their

interests, and they try to prevent action when it is not in their interests. So you

don’t have a universal, much less a completely equitable, enforcement regime,

but one that is just as subject to the vagaries of politics as anything else. If the

United Nations had an army, and the army was effective, then the United

Nations might be able to enforce its own agenda of human rights, but even

Human Rights: A Very Bad Idea 87

that would not get you out of the realm of politics and positive rights because

then the United Nations would be something like a super-state and those rights

they enforced would be positive (not merely natural) rights. In fact, as you

know, what happens is that the United Nations passes resolution after resolution

about human rights, human rights violations in one place or another, and

they are acted on only if the United States and the Western powers find it in

their interest to do so. So, they invade Iraq when it suits them; they didn’t

invade Iraq, actually, when it didn’t suit them. The human rights record was

the same in both cases: Saddam Hussein was ‘violating human rights’ for

decades, the West had no interest in dealing with that at all. Then, when Saddam

began to show too much independence, suddenly the U.S. seemed to

acquire an interest in his human rights record, so suddenly, instead of continuing

to give him arms to fight against the Iranians they invade his country!

The actual politics of human rights on the international scene is the politics of

giving powerful countries the moral justification to do what they’d like to do

anyway under the specious cover of enforcing human rights. If the United

States doesn’t want to invade Iraq, whatever the United Nations says, nothing

happens. The United States does want to invade Iraq, and then suddenly it is

not just an act of military aggression, but one covered by the moral cloak of

enforcing human rights.

That is one of the things that worries me about the talk about human rights

in the international realm, that in fact, human rights in the international realm

are a legitimating cover for Western powers enforcing their interests and there

I think it would be more honest for us simply to say, ‘well, the U.S. didn’t like

Saddam Hussein at that point, so they invaded and deposed him’ rather than

saying they suddenly began enforcing a regime of human rights.

Lawrence Hamilton: I agree with that and I think that everyone on the editorial

board would as well, but, what of the slightly distinct claim that – and

we’ll come to this in a bit more detail as well in respect to particular nation

states – what of the slightly distinct claim that it might not always simply be a

tool of the powerful but also the less powerful, that it might in some instances

be utilised, because it’s a norm, illusory or otherwise, as an effective means of

political agency? So there are two parts to the question, the first part is that,

even though human rights may be an instance of white magic and puffery in

the way that you are suggesting plus the fact that they legitimate the actions of

the powerful etcetera in certain kinds of moralising ways, they’re so

entrenched in the discourse, that do you think it is realistically possible to simply

wish them away? That’s the first part; I’ll ask the second in a moment.

Raymond Geuss: All sorts of things can have positive or negative rhetorical,

or real, effects in politics. I don’t deny that in some contexts people can say

‘we are rising up, because it’s a violation of our human rights’. But you could

equally imagine the same group of people saying ‘we are rising up because

88 Raymond Geuss, interviewed by Lawrence Hamilton

we’re human beings’, ‘we are rising up because we are children of God’, ‘we

are rising up because our needs are not being met’, ‘we are rising up because

we have had bad dreams’, ‘we are rising up because we have had good dreams’

etc. There are all sorts of different forms that the structuration of resistance

can take: it can take theological forms, it can take ethnic forms, it can take

political forms, it can take class-based forms, and so on. In certain contexts it

might be effective and useful to appeal to human rights. After all, ‘human

rights’ have no more, but also no less cognitive and normative standing than

dreams do.

As political theorists, however, it behoves us not just to look at the way

demands are actually formatted and expressed, and see whether or not they

are rhetorically effective, but to look also at the internal structure of the

demands, the specific way that demand is expressed, and the theoretical implications

of that form of expression. So we may demand education because education

is an important way of our satisfying our interests. Or we say we have a

need for it. That is a more perspicuous way of saying something important

than by speaking of ‘rights’. By calling it a right you don’t add anything to

calling it a satisfaction of a basic need or an important demand or a vital

human interest. You not only don’t add anything, rather you say the same thing

in a format that has a history and carries with it a historical legacy of additional

connotations. That legacy is not necessarily unmitigatedly good. If, on the

other hand, you talk about education as being an important thing for people,

as being a human need, this way of speaking moves you on constructively to

an interesting political discussion. If I say it is an important need that I have

an education, then you ask me ‘why is it a need?’, then I will talk about my

needs, and this will naturally lead to discussion of other aspects of my life.

That’s a political way of formulating the demand. If I say ‘it’s a right’ I’m

immediately saying it’s a trump, that is something I don’t discuss, it’s something

that’s limited, it’s something that’s rigid and determinate, it’s something

to stop the political discussion, it’s inviolable, and so on. In some contexts it

might be good to be irrational and to formulate your demands in particularly

rigid, and particularly non-negotiable ways, but it doesn’t follow from that

that would be politically the most interesting and intelligent way of understanding

what’s really going on.

Lawrence Hamilton: Even if it were possible, let’s imagine, do you think

human rights can track human interests?

Raymond Geuss: Of course they can track interests; under appropriate circumstances.

However, by immediately moving to the level of rights-discourse,

you cut off discussion of whether the circumstances are appropriate. After all,

in some circumstances theological discourse can track interests, too. When

the Catholic says ‘God wants every man to have his daily bread’, of course,

every man should have his daily bread, and the theological discourse tells us a

Human Rights: A Very Bad Idea 89

truth! [Laughter] I’m not claiming that it can never be the case that speaking

of rights can be connected, or can be interpreted, in a way that makes that

mode of speaking make sense. I’m also not saying that this way of speaking

can never be effective. Sometimes it is effective, sometimes not; sometimes it

does track interests, sometimes not. However, as theorists, we have not just to

decide whether there is something in a particular form of speech that’s right

and whether this way of speaking is rhetorically effective in a particular context,

we have to understand also whether what is right is perspicuously formulated,

and when I say ‘God says: give every man his daily bread’ that’s not a

perspicuous formulation of an important demand. It is a presentation of something

that is a true human interest or vital human need in theological vocabulary.

Sometimes I can give a translation, but why go down the detour of

formulating human interests through rights-talk, and then have to translate

what you’ve said back into something that makes more sense? Just talk instead

about interests, wishes, demands, needs, what we think is good for humans,

what is outrageous or intolerable, etc.

Again, as political theorists we have to operate on two levels at the same

time. You have to act on the level of what is actually effective, what will get

people going, what will motivate them, what is the common discourse in

use. You have to take that seriously. But you also have to take seriously that

that’s not the final word: sometimes everyday modes of speech are correct

and enlightening and sometimes not. Being a serious political theorist means

accepting both those things, that is, accepting that you can’t simply ignore

what people say in the name of what you know is true, but you also can’t

simply always accept what people say at face-value. You can’t just accept

what they say and ignore what’s true, but you also can’t just say what’s true,

while ignoring what they think and say. You have to have both of those

aspects and you have to connect them in some appropriate way. Rights discourse

stops that or makes it more difficult and for precisely that reason: it’s

a trap, to be more exact, it is a trap because it tries to construe political situations

as apolitical.

If rights are trumps, that is, they’re supposed to stop discussion. Another

way to say that is that they’re unpolitical. They’re an attempt to shift from politics

into legalism, to give a legal model for politics. They’re also rigid, they’re

forms of rigidity. If they are supposed to hold for everyone in all situations,

they can’t by their very nature take account of the peculiarities of individual

situations. And in some sense all political situations are individual. Finally,

they are also guilty of putting different things at the same level: a right to life,

a right to education, a right to entertainment. These are not necessarily things

on the same level in human life or in politics. In politics we often have to think

about the differential importance of things, to say there’s nothing but rights,

however, levels out significant qualitative differences.

Lawrence Hamilton: In politics one needs to set priorities.

90 Raymond Geuss, interviewed by Lawrence Hamilton

Raymond Geuss: To set priorities, to have conflicts, to override some claims

in particular cases, that’s what politics is about. Rights-discourse tries to stop

that kind of debates on those issues. So it’s an inherently apolitical way of

thinking about politics. It’s a particularly clumsy, crude and rigid attempt to

turn politics into a kind of administration or legal discussion. One can understand

why it is tempting to think about human life in that way, and why in

some context it might even be important to think about some limited matters

in this way. But it’s not the right way finally to think about politics.

Lawrence Hamilton: So, let’s take one of those examples. The one that comes

to my mind, and to lots of others, is the case of South Africa. I’m going to

jump ahead to this example and then come back to some of the more theoretical

matters later. There are actually two really nice examples, I think, about

the way in which the language of human rights under certain conditions, not

just its effectiveness, but other components of it, might be highly laudable.

First, you ask in History and Illusion in Politics that if ‘irresolvable conflict

between bearers of purported equal rights is thoroughgoing and unavoidable,

what is the point of recourse to rights?’ (p. 149) The argument is well taken,

but what of this response? Under conditions of real or potential civil conflict,

the language of human rights provides a common, otherwise absent, sense of

legal and moral norm that can form the motivational basis around which those

on the verge of conflict can cohere. Rather than suppose that human rights

only works or ‘works on us’ as a foolproof legal and moral structure – and this

comes back to your own point – is it not more realistic and helpful to see them

as a philosophically and theoretically rough-and-ready and even incoherent

tool – but the best we have – to assure arch-enemies that their interests will

not be forgotten once they give-up their weapons? South Africa just prior to

and following the release of Nelson Mandela is a fabulous case in point, in the

sense that without the language of human rights, many have argued, it would

probably have been very unlikely for two extremely opposing groups, the

National Party and the African National Congress (ANC), to cohere over some

common ground.

Raymond Geuss: I have two responses for that. One is, I don’t mean to deny

that the language of human rights might have played an important role actually

in that situation. But I invite you to think about whether the same function

might have been played by things like the common good, the common interest,

all of our needs, our collective needs, social solidarity .... Aren’t there other

ways of talking, aren’t there other possible ways of talking about societies

that are deeply divided, which would give you what you want. I admire the

aspiration to universality in the notion of human rights. I’m not objecting to

an aspiration to a certain kind of universality, but what I’m suggesting is,

there are other ways of attaining that universality. You can think about it in

terms of the common good, you can think about it in terms of a society con-

Human Rights: A Very Bad Idea 91

stituting an encompassing whole, you can think about it in terms of forms of

participation, of people universally participating, you can think about it in

terms of mutual vulnerability, you can think about it in terms of reciprocal

dependence. I don’t see there is anything added by the language of universal

rights or human rights, anything specifically added that could not be

expressed in those ways, and those things would have the extra advantage that

they wouldn’t commit you to a quasi-juridical framework. It would make it

clear that if a society is going to hold together it has to be because of social

and political processes that are ongoing.

Rights are not processes. Rights are trumps, they stop the process. Now in

some contexts that might be the best you can get. However politics is about

processes, and our aspiration should be to make those processes as participatory

as possible. What we have to think about is the notion of collective forms

of agency, collective powers, and collective forms of suffering. And to talk

about rights is to take attention away from these other important things, it’s to

take attention away from who the agencies will be, and how co-operation will

be structured. It’s to take our attention away from finding forms of living

together which won’t require giving trumps to people. All of those things are

part of the positive tasks of politics. Perhaps in some context the appeal to

human rights might have been useful. But I think from the fact that it was useful

in that context, it doesn’t follow that there weren’t other ways you could

think about the demands put forward, and it doesn’t follow that there were not

more productive ways to think about these demands, their origin, and the ways

in which they might be attended to and satisfied.

Lawrence Hamilton: And in the South African case as well, since the great

‘miracle’ of the relatively peaceful end to the apartheid regime, the strict and

strong human rights nature of much of our legal and political system is inhibiting

in various kinds of ways. So, post a situation in which South Africa

avoided civil war, where your politics has become highly concretised in the

form of human rights, you start to come across various problems when politics

is carried on in terms of human rights.

Raymond Geuss: My suggestion for the replacement for human rights is

politics.

[Laughter]

Good politics. Think about what good politics is. Some topics should be:

human agency, human interests, human powers, human needs – That’s one of

your big topics, Lawrence – ways of negotiating boundaries between people,

ways of producing new structures of co-operation. All of these seem to me to

be important and thinking about them seems a much more fruitful and forward-

looking way of approaching politics, than through obsession with rights.

Formatting everything through human rights has a stultifying effect on

thought and action. That’s compatible with saying in certain domains and cer-

92 Raymond Geuss, interviewed by Lawrence Hamilton

tain times human rights might be a useful thing, that is, however, something of

very limited use or interest. Rights-obsession has a stultifying effect because

it takes attention away from these underlying processes of the formation of

collective agencies and collective satisfaction of needs and interests.

Lawrence Hamilton: In a place like South Africa where executive and legislative

power, even if formally represented, leaves a lot to be desired both in

terms of representation and good government what really is the problem of

each of us insisting on his or her own rights without the mediation of lawyers?

There may be an element of wishful thinking here, as in a politics so bound up

with rights, we cannot effectively do so without legal representatives, but even

if at some point in the process the bureaucratic services of lawyers are needed,

is not the process of claiming and exercising rights an important form of political

agency for modern citizens? If we did away with it, what form of political

agency might we have beside that provided by periodic elections?

Raymond Geuss: That’s exactly the opposite of what I’m trying to say! What

that says is, look, we’ve got two forms of political activity: appeal to human

rights and elections. What I’m trying to say, however, is if you think about

politics as appealing to human rights and participating in elections, you’ve

got a remarkably thinned out and impoverished notion of what politics is supposed

to be about. If you say what politics should be, is, we elect people and

we claim our rights, you’ve lost the game already. Elections are very crude

and very rudimentary forms of political activity.

A country that’s in a healthy state should have all sorts of forms of political

agency that are not mediated through the election of formal representatives.

There should be a wide variety of different associations and ways of doing

politics. To give priority to claiming rights gives us the completely wrong attitude

toward ourselves, other political agencies and the state because it basically

prioritises us as passive clients of the nanny-state. This idea of a

‘nanny-state’ is a conservative invention, but what is wrong with it is the tacit

(and utterly implausible form of) atomistic individualism that stands behind it

and the irrational suspicion of any form of collective organisation. It does,

however, contain the germ of a correct perception about the deleterious effects

of the potential passivity of modern populations. If all we do is vote, and then

shout for mama whenever someone treads on our toes, saying ‘This is my

land; I claim this right, I claim that right, I claim the other right’, that is a

really very narrow way of thinking about what politics can do and be. And if

you take that as your central way of thinking about politics you’re going to

have a tremendously distorted and impoverished form of politics.

Lawrence Hamilton: What of the liberal concern, though, about the capacities

and virtue of political representatives, which emerge even in quite secure and

common situations generally, but seem to be worse in a place like South Africa,

Human Rights: A Very Bad Idea 93

where you’d want some kind of mechanism of control over your representatives.

The liberal argument, as you know, is that rights are the best form we

know of for controlling the powers and actions of representatives and the state?

Raymond Geuss:Well, if you think that that’s the best way of controlling the

representatives and the actions of the state you’re not actually going to be

motivated to look for new forms of control, and you’re going to have with that

all of the problems that I think I’ve tried to mention about rights. For instance,

different rights that belong to different domains and have different priorities

will be put on the same level. You’re going to have people struggling to get

something to eat, and people with property rights, and the right of the land

owner to his land is going to put on the same level as the right of the poor individual

who is starving.

So, yes, it is important to have protections for people – protections in certain

limited domains – and it might be appropriate for those protections to

take the form of rights, but that’s different from saying that the basic structure

of the society and politics should be thought of through rights. Rights will be

very particular, limited sorts of things which will be put in force in the context

of a larger framework and larger scheme of politics, and if you’re really interested

in politics in the most important sense you’ll be interested in the big

scheme, you won’t be interested in the tiny mechanisms that are put in to protect

individuals against certain abuses.

Lawrence Hamilton: And in such situations – again it’s not unique to South

Africa it’s just starkly manifest in a place like South Africa – where something

like what you’ve just described, as you know from my own work, is a very

serious problem, where a very distorted history generates situations in which

an existing right to land comes up against a right of access to land, and it’s

very difficult to resolve the dispute in terms of rights. But what of a situation

in which the notion of human rights and human rights in general have come to

underpin the legal system of the nation state? In these situations, therefore,

human rights in the objective sense, in your objective sense, have become

human rights in the subjective sense. In these circumstances do you think it’s

a good idea to try and undo these achievements? And, relatedly, returning to

an earlier point, why not acknowledge human rights as ideological constructs

and just try and foster good ones, or better ones?

Raymond Geuss: I don’t see that there’s any particular necessity to dismantle

the existing constitutional arrangements concerning rights. Maybe we should

just ignore them. In any case I’m trying to talk to other people who are interested

in understanding society in some non-trivial way, not just in operating or

tinkering with the existing mechanisms of government. The existing political

mechanism in Britain has all sorts of parts that we wouldn’t take seriously if

they were not historically already entrenched.

94 Raymond Geuss, interviewed by Lawrence Hamilton

Now it might be the case that we continue on with various practices even

when we have seen through the purported rationale for these practices. For

example, in court it used to be the case you swore on the Bible to tell the truth.

Now as an atheist you don’t need to swear, but you also don’t even need to

object to being required to swear on the Bible to tell the truth, because you

interpreted that as merely meaning ‘I now assert in a solemn way that I am

going to tell the truth’. If people want to call that ‘swearing to God’, let them.

I feel no need to oppose that even if I think God doesn’t exist. So I don’t think

you have to be a purist and go through all the legal codes to get rid of all the

references to rights. However, as a political philosopher you have to think

about what rights-discourse actually means and I think if you begin to think

about that you’ll begin to be more impressed by the limitations of that way of

thinking about things. What was the second question again?

Lawrence Hamilton: The second question is: why not simply view human

rights as ideological constructs and foster good ones? Why assume that it must

be the ‘bad guys’, as it were, that are always going to be the ones constructing

and abusing rights and human rights, why can’t human rights be in the hands

of the ‘good guys’?

Raymond Geuss:Well, that very way of putting it seems to me to be grist for

my mill, because what you’re saying is the regime of human rights in itself is

neither good nor bad. What’s good or bad is the people who are using it for

one purpose rather than another, and it seems to me that’s another way of saying

something, which, if you think about it seriously, will be likely to move

you in my direction.

It will be likely that you’ll be motivated to say that what’s important is not,

as it were, the form of the scheme of rights, but the political process within

which it is embedded, and that’s what I’ve been trying to encourage people to

do: to look at the political process within which ‘rights’ are embedded and

then to ask whether it’s good to restrict that political process through a regime

of rights or not. And if you begin to think about it that way, you’re already

thinking about it differently from the way in which traditional human rights

people thought about it. They thought ‘These are rights, they’re rooted in

human nature, they belong to all people; once you get them set out and embedded,

you don’t have to worry, as it were, about who’s going to enforce them or

what the political processes are’.

Lawrence Hamilton: And, what would you say to a believer, who believes

not just in God, but that God imbued us as humans with natural or human

rights?

Raymond Geuss: That’s a perfectly coherent view. Perfectly coherent, I think,

but false. However, if you believe that there is a God, and you believe that that

Human Rights: A Very Bad Idea 95

God is interested in human beings, and if you believe that that God has a particular

purpose for human beings, then you have good reasons to believe in

human rights as part of that divine purpose. I just happen to think that those

assumptions are false!

Lawrence Hamilton: And if you have a society in which the majority of people

still believe that, then what?

Raymond Geuss: Then I come back to my other point about witchcraft and

the example of the Early Modern period. If you have a society where everyone

believes in God and believes these rights have been given by God – then as a

political theorist you have to have a double perspective on that society. You

have to look at that society as it actually operates, the way it operates through

the beliefs of the people, the way in which the beliefs of the people animate

the social system and the system of rights, and then also you have to take a

perspective outside that of the society and look at what’s going on in the society

from your own point of view. Finally you have to try to put those two

things together synoptically; I don’t think that’s impossible. I can talk about a

society within which everyone believes in God and I can then analyse why in

that society some set of rights will work as a good way of organising it without

committing myself to the existence of that God or of these rights.

Lawrence Hamilton: So, in that case, you’re not going to be able to provide

a motivation, an alternative motivational structure?

Raymond Geuss: Now we’re moving to another question, not a question

about the justification of rights, or about what reality these rights have. Rather

we’re moving to the question raised by the example of a society in which I am

thought to be a witchdoctor. Everyone thinks I have these powers. In that society

what new motivations could I give to people? It is a very important question,

how you give people motivation for things. But note, if you start by

saying ‘I’m going to give people new motivations for things’ why do you then

stop and say ‘I’m not going to give them new motivations to be politically participatory’

or new motivations to think about and try to pursue the common

interest, or new motivations to invent new collective forms of agency? It seems

a bit tame at that point to say ‘I’m merely going to give them new motivations

to instantiate a regime of rights’. The project of giving people new motivations

seems potentially more ambitious than simply appealing to rights … Another

way of putting what I’m trying to say is that thinking about the society in

terms of rights is not thinking about giving people new motivations. It’s thinking

about maintaining the status quo and structures associated with that status

quo and, at best, trying to bend these in a progressive way. What one should

actually be doing is trying to give people new forms of motivation. You’ll say,

of course – and it is quite correct – that that task is really hard. My reply to

96 Raymond Geuss, interviewed by Lawrence Hamilton

that is that politics is really hard, and the real question is what the aspirations

for your politics are. Do you have low aspirations for your politics, or high

aspirations? To accept the regime of rights as the existing framework is to

have low aspirations.

Politics should have a higher aspiration than that, and part of the aspiration

might be to give people motivations that are independent of the language of

rights. So, if you create new forms of cooperation, these new forms of cooperation

will give people new forms of motivations which won’t be connected

necessarily with the rights discourse.

Lawrence Hamilton: That’s really fascinating, but just playing Devil’s advocate

here, and imagining the rights activist responding to that, and there’s a

great deal of literature on this, might you not be guilty of making too strict a

distinction? There’s a lot of history, not just in a place like South Africa, but

worldwide, on the fact that human rights discourse does generate a great deal

of participation in politics, it does generate a sense of the collective, it does

generate a sense of political agency in various forms that didn’t exist prior to

the prevalence of the notion of human rights.

Raymond Geuss: I’m sorry Lawrence, I don’t quite know how to respond to

that question because you seem to be saying something that is clearly true but

I don’t see why it’s a problem for me.

All sorts of things can give people motivations, appeals to Islam can give

people motivation, appeals to Christianity, appeals to the class struggle,

appeals to Reason. There are all sorts of things that can give people motivation.

Rights discourse in the right context can give them motivation too. I’m not

denying that we should try to give people motivation.

I’m saying two things: First, it’s a bad idea to give them motivation in terms

of rights. Whatever motivation they can be given through rights discourse could

be given to them through something else which is less obfuscating and mystifying,

such as appeal to their collective interest, appeal to their needs, appeal to

various sorts of things. Second, we should actually be trying to appeal to those

other things because that will get us a better kind of politics, it will get us a politics

that’s more active, where the people are active and engaged, less rigid, and

less tied to legalistic forms of acting. How can we find ways of making people

active and not reduce politics to their simply demanding rights?

Lawrence Hamilton: And the language of rights is inhibitive of that process?

Raymond Geuss: Rights by their very nature must be crude. They can’t be

subtle or highly differentiated because they’re legal functions. They have to

be relatively undifferentiated and rough-hewn, but politics should be about

making people more differentiated. It shouldn’t be about just making blocklike

demands, such as: ‘protect my property’, it should be about thinking about

Human Rights: A Very Bad Idea 97

what in this city is the right rate of investment in education, in health; how we

should structure new forms of urban development; where we should build

schools and hospitals. You can’t reduce that to mere demands for rights.

Legal structures have to be coarse-grained because they have to refer to

processes that are verifiably repeatable and that are subject to public scrutiny.

Political process should try to be more differentiated than that and politics

should be about something more than merely claiming rights. Politics, as I

have said, should aim at activating people.

Lawrence Hamilton: Needless to say I share the…

Raymond Geuss: The goal.

Lawrence Hamilton: And also the desire to have the citizenry as a whole

more politically active and more understanding of the political process but

there are very many thinkers and activists – not just liberals – who will say the

crude nature of something like human rights and rights in general befits politics

because one can’t really expect that level of understanding or participation

or judgement.

Raymond Geuss: Ok, but note what you’re saying there. What you’re saying

is rights discourse is a second best. It’s at best a crude approximation. It might

be – as in the case of South Africa on the verge of civil war – that in certain

emergency situations we must use blunt instruments that do no more than

merely work. However, as political actors and as political theorists we have to

have the ability not just to think in terms of second-best solutions, we have to

allow ourselves the possibility or the hope that the citizens can be brought to

be slightly less primitive in their reaction to politics, that the citizenry can

learn to think beyond that, and that’s discouraged by the assumption that

everything has to fit in the category ‘human rights’.

Lawrence Hamilton: So, to leap to something else: would you see this as

part of a way in which you can think realistically in a utopian fashion, as it

were, and how does that link to your later argument in Politics and the Imagination

(Princeton UP 2009) regarding meridian and constellation?

Raymond Geuss: Once you leave behind the view that politics is about rights,

that might possibly open a number of doors to thinking about politics in a

number of different ways and I’ve tried to think about some particular literary

figures and some older thinkers in the tradition of philosophy who tried to

develop alternative ways of approaching some aspects of politics. Meridian

and constellation are one or two such concepts that I use. It isn’t that I think

that they’re the only ones, it’s that I think that they represent a kind of gesture

in the right direction.

98 Raymond Geuss, interviewed by Lawrence Hamilton

Lawrence Hamilton: Can you say a little more about meridian.

Raymond Geuss: That comes from this…poet…

Lawrence Hamilton: Paul Celan.

Raymond Geuss: ... A meridian is an imaginary line connecting two places

relative to the position of the sun. Celan thought a meridian was a way of

locating oneself in the world by reference to concrete places, people and

things. That’s a different way of thinking about the world from thinking about

the world through abstract rights categories, so that’s a way of trying to make

political thinking more concrete and more realistic. What is the meridian here?

On what meridian are we standing at the moment? Who are the people that

we’re working with, who are the people who have been our traditional friends

(or, enemies), what can we do, how are we located relative to them? All of

those things are things you can think about which you can think about in ways

which seem much more illuminating if you don’t think about yourself, your

friends, your allies, your potential allies and your enemies in terms of categories

of human rights. You think about them more in terms of concrete historical

or concrete geographic relations you might have to them.

Lawrence Hamilton: So it’s not intended as an alternative mechanism for

doing politics, it’s intended as an alternative mechanism for thinking about

politics?

Raymond Geuss: Yes, yes. First one might connect it with the Gramscian

idea of the importance of forming political alliances. One might try to form

such alliances along meridians, for example. Gramsci, of course, was very

keen on this notion that politics is importantly about who we make an alliance

with and who we don’t make an alliance with. So we might use some of the

spatial and temporal metaphors to talk about which groups you will make

alliances with, what their historical relations to one another have been, what

possible consequences of making an alliance with that concrete group will be.

Lawrence Hamilton: Which would mean, given different concrete situations

outside and across nation state boundaries and the norms and normal ways in

which we’ve thought about politics, it might start to look a bit like something

in which there exist groups and interests along class lines, for example, that

we start to conceive as paramount for politics.

Raymond Geuss: For example. Yes, for example. The idea is orienting

yourself to concrete others, concrete other places, concrete other groups

rather than orienting yourself to abstract categories like ‘person’, ‘human

beings’ etc.

Human Rights: A Very Bad Idea 99

Lawrence Hamilton: And concrete problems associated with …

Raymond Geuss: And concrete problems associated with … exactly.

Lawrence Hamilton: Wonderful. We’ve leapt ahead to the end, so I’m just

going to come back to one or two more particular questions. You say in History

and Illusion in Politics that the status, or ‘cash value’, of rights is really just:

‘we think that it would be a good idea for there to be a reliable system of

effective power to enforce X’s claim to Y (even if there is not such a system)’.

Even if there is not such a system, you then argue, that even if most people do

come to agree on a shared conception of human rights, it still does not follow

that even shared beliefs reliably translate themselves to predictable, appropriate

action, and that is what is at issue. Why? Might not a shared conception of

human rights reliably translate into predictably appropriate action with either

positive or perverse outcomes: such as driving on the right side or on the same

side of the road, or the other example given …

Raymond Geuss: Abortion rights.

Lawrence Hamilton: Abortion rights and foetus rights: you might have a situation

in which legally it would be very easy to get an abortion but you might

have a situation in which there’s a, as you put it, a reliably large number of

people who are influenced by the notion of foetus rights.

Raymond Geuss: This seems to me to be another instance of the magician

who is purportedly able to protect his property by virtue of the people thinking

that he has magical powers. If enough people believe it, that belief will

have self-reinforcing effects and might reliably bring about certain outcomes

which might be good or might be bad; but what I insist on is that as theorists

we have to understand that that’s the nature of what’s happening. The magician’s

power is not ‘natural’; it isn’t that driving on the left or on the right is

natural. These are both social facts and I should recognise that they have been

brought into existence by people thinking and acting in a certain way, just as

I recognise that people have brought about regimes of rights. However, from

the fact that they have brought those regimes of rights into existence and

enforce them it doesn’t follow that those regimes of rights are natural or

innate or objective.

Suppose everyone in the world agreed on a budget of human rights, and we

all enforced it. Then, of course, we would be enforcing a regime of rights. I’m

not denying that. That would be foolish; but what I’m denying is that even if

we universally agreed on a budget of human rights and we enforced them,

they would not (thereby) be natural, objective, innate, rooted in our mere

humanity, etc. Rather they would be rights that we gave to people and

enforced; they wouldn’t be rights that were out there naturally. Just as if we all

100 Raymond Geuss, interviewed by Lawrence Hamilton

agreed to drive on the left, we drive on the left because we’ve all agreed to do

it. Not because it’s natural to drive on the left.

Lawrence Hamilton: This brings us back neatly to the opposing view which

is that the intellectual history or the emphasis on the natural rights foundation

of human rights is misplaced. In actual fact, the important component of

human rights is their cultural-political history in which individuals out there,

citizens out there, participants out there, have been claiming, in this case in

terms of human rights, for certain goods.

Raymond Geuss: If people want to go out on the street and protest in the

name of human rights, so be it. What is important is the content of the right,

the mechanism for ascription and enforcement, and clarity about what it is

one is doing (and what one is not doing).

Lawrence Hamilton: The only objection you have is that they then claim that

to be a kind of natural right.

Raymond Geuss: I have an objection if they claim that to be natural rights. I

also have an objection to the idea that the way to think about politics finally is

through the instantiation of specific rights – whether natural or positive – and

the enforcement of specific rights, rather than in terms of broader political

processes. What I’m objecting to is the idea that politics can be universally

formatted in terms of human rights.

Lawrence Hamilton: And of course, in the West they tend to be, right? And

in other places they tend not to be.

Raymond Geuss: In some other places they don’t.

Lawrence Hamilton: Like China. So, a thought that comes to mind is, given the

world we have, might not one possible way of resolving the problem be some

kind of ideological or even military conflict between the opposing positions?

Raymond Geuss:Wars generally have less effete origins than ideological differences

of opinion; they come from a real clash of interests. So, part of my

immediate reaction to that would be to think that this worry depends on a very

idealistic theory of history, thinking of history in terms of the struggle between

ideological forces. It’s a sort of – what’s the guy’s name? Huntington – a

Clash of Civilizations thing and I’m afraid I’m too old fashioned, too Marxist

still to think that that’s the right way to approach things.

It isn’t civilisations that clash. It isn’t ideological constructions that clash.

Or, rather, they can ‘clash’ all they want, but no one needs pay too much attention.

It’s concrete agents including state agents who have interests and powers,

Human Rights: A Very Bad Idea 101

who conduct wars. So I wouldn’t think that it was inevitable in any way that

because the Chinese don’t have a concept of human rights, and most people in

the West have a concept of human rights, there will necessarily be a war. It

doesn’t follow from that that I don’t think there will be other difficulties and

struggles, certainly economic struggles and perhaps also military conflict, but

such conflicts will not be best construed as motivated by ideological differences.

They’ll be motivated by such things as the water supply in South-East

Asia. The control of the Himalayan area is geo-politically extremely important

because a lot of the water resources for Southern Asia arise in that area – in

Tibet – and with population pressure and changes in the climate, the border

between India and China is, I think, going to be increasingly a region of tension.

So there is a possibility of some kind of conflict between Pakistan, China

and India in that border region and that seems more likely than direct conflict

between China and the United States. But of course that doesn’t exclude the

fact that China and the United States won’t be drawn into conflict, but that it

won’t be on these ideological grounds. Now, to be sure, the Americans will

claim, once any war starts, that it’s an ideological war …

Lawrence Hamilton: They’ll claim that the human right to water…

Raymond Geuss: They may well claim that it’s about human rights just as

they claimed that the war in Iraq was about ending tyranny. But we’re not

going to be taken in by that are we? Sophisticated people that we are.

[Laughter]

Lawrence Hamilton: So perhaps all talk of rights is an inconvenient fiction.

Raymond Geuss: Crude and inconvenient.

Lawrence Hamilton: Crude and inconvenient fiction. But we … why do we

appear to need this fiction?

Raymond Geuss: That’s a very good question.

Lawrence Hamilton: You give the beginnings of an answer, it seems, in History

and Illusion in Politics, where you talk about efficiency, predictability,

security? But, might you want to say more than that? For example, I don’t want

to put words in your mouth, but one potential way of thinking about this would

be to say, and it links to your argument about Iraq, would be to say – would you

want also to argue – that human rights are primarily useful for the reproduction

of those ruling classes who currently control the state in any particular polity?

Raymond Geuss: I wish I had a good answer to that. I don’t but I’ll say what

I can. I think there are three sources for this fascination with rights. One is the

102 Raymond Geuss, interviewed by Lawrence Hamilton

capitalist economy which depends on predictable control over resources, it

depends on knowing who owns what. The model of all rights, after all, is a

property right and that’s the thing we must never forget. Rights come out of

property thinking: who controls what property? That’s the central thing. And

so you can understand that a capitalist society will be absolutely obsessed

with clarifying who owns what factors as clearly as possible, so that will be

part of the basic apparatus of thinking in such a society.

Second, there is a Foucauldian element, which is that the state wants to get

control over its citizens and so there will be complicated processes of individuating

them and keeping them under control. The relation of this to rights-discourse

is complex and obscure, but I think a connection exists.

Third, in societies in which people have actually little control politically

over what happens, individuals will have compensatory needs because they

feel powerless. They will want to think that ‘well, at least I have some domain

in which I have control, I might not be able to control what happens to my

city, what happens to my street, what happens in the hospital, what happens in

the war, but at least I have the right to something-or-other.’ Rights under this

aspect are the modern form of opiate for politically neutred populations.

So I think it’s a combination of the needs of the economy, the needs of a

modern administrative state and the psychological compensatory needs of

deracinated populations that are relatively alienated and feel themselves to be

powerless and rights-thinking is a kind of way in which they can feel good

about themselves. They can feel: although we have no control over things, at

least in this domain we’ve got these rights.

Lawrence Hamilton: Excellent. Thank you Raymond, that’s a wonderful

place to stop partly because it speaks directly to the South African situation,

that last point in particular. I think that besides the point I made earlier about

cohering sharply divided interests and histories – the importance of getting

two very divergent groups together – the function it performs in a place like

South Africa is the compensatory function you’ve just articulated. That was

fascinating, thank you!

Raymond Geuss: Good, good.

Lawrence Hamilton would like to thank François Janse van Rensburg and the

editorial board of Theoria for editorial assistance in producing this published

interview.

Human Rights: A Very Bad Idea 103

Friday, January 17, 2020

George Hotz: Comma.ai, OpenPilot, and Autonomous Vehicles | Artificial Intelligence (AI) Podcast

George Hotz: Comma.ai, OpenPilot, and Autonomous Vehicles | Artificial Intelligence (AI) Podcast


the following is a conversation with George Hotz he's the founder of comma AI a machine learning based vehicle automation company he is most certainly an outspoken personality in the field of AI and technology in general he first gained recognition for being the first person to carry or unlock an iPhone and since then he's done quite a few interesting things at the intersection of hardware and software this is the artificial intelligence podcast if you enjoy it subscribe on YouTube give it five stars on iTunes supported on patreon or simply connect with me on Twitter at lex friedman spelled fri d-m a.m. and i'd like to give a special thank you to Jennifer from Canada for her support of the podcast on patreon merci beaucoup Jennifer she's been a friend and an engineering colleague for many years since I was in grad school your support means a lot and inspires me to keep this series going and now here's my conversation with George Hotz do you think we're living in a simulation yes but it may be unfalsifiable what do you mean by unfalsifiable so if the simulation is designed in such a way that they did like a formal proof to show that no information can get in and out and if their hardware is designed to for the anything in the simulation to always keep the hardware in spec it may be impossible to prove whether we're in a simulation or not so they've designed it such there's the closed system you can't get outside of the system well maybe it's one of three worlds we're either in a simulation which can be exploited we're in a simulation which not only can't be exploited but like the same things too about VMs I'm a really well-designed VM you can't even detect if you're in a VM or not that's brilliant so where it's yeah so the simulation is running in a virtual machine but now in reality all VMs have wasted the fact that's the point I mean is it yeah you've done quite a bit of hacking yourself and so you should know that really any complicated system will have ways in and out so this isn't necessarily true going forward I spent my time away from comma I learned cock and said dependently typed like it's a language for writing math proofs and if you write code that compiles in a language like that it is correct by definition the types check it's correct and so it's possible that the simulation is written in a language like this in which case yeah yeah but that can't be sufficiently expressive a language like that all weekend it can be yeah okay well so all right so the simulation doesn't have to be tiring complete if it has a scheduled end date looks like it does actually with entropy and you know I don't think that a simulation that results in something as complicated in universe would have a formal proof of correctness right as as possible of course we have no idea how good their tooling is and we have no idea how complicated the universe computer really is it may be quite simple it's just very large right it's very it's definitely very large but the fundamental rules might be super simple yeah Conway's gonna live kind of stuff right so if you could hack it so imagine the simulation that is hackable if you could hack it what would you change about the you know like how would you approach hacking a simulation the reason I gave that talk I by the way I'm not familiar with the talk he gave I just read that you talked about escaping the simulation yeah like that so maybe you can tell me a little bit about the theme and the message there - it wasn't a very practical talk about how to actually escape a simulation it was more about a way of restructuring an us-versus-them narrative if we continue on the path we're going with technology I think we're in big trouble like as a species and not just as a species but even as me as an individual member of the species so if we could change rhetoric to be more like to think upwards like to think about that we're in a simulation and how we could get out already we'd be on the right path what you actually do once you do that while I assume I would have acquired way more intelligence in the process of doing that so I'll just ask that so the the thinking upwards what kind of ideas what kind of breakthrough ideas do you think thinking in that way could inspire and what did you say upwards upwards into space are you thinking sort of exploration in all forms the space narrative that held for the modernist generation doesn't hold as well for the postmodern generation what's the space narrator we're talking about the same space the dimensional space like going a little ace is like building like yuan mosque like we're gonna build rockets we're gonna go to Mars we're gonna colonize the universe and the narrative your friend was born in the Soviet Union you're referring to the race to space the race to space explore okay that was a great modernist narrative it doesn't seem to hold the same weight in today's culture I'm hoping for good postmodern narratives that replace it so think let's think so you work a lot with AI so the eyes one formulation of that narrative there could be also I don't know how much you do in VR and they are yeah that's another eye I know less about it but every time I play with it and our research is fascinating that virtual world are you are you interested in the virtual world I would like to move to a virtual reality in terms of your work no I would like to physically move there the apartment I can rent in the cloud is way better in the apartment I can rent in the real world well it's all relative isn't it because others will have very nice departments too so you'll be inferior in the virtual world that's not how I view the world right I don't view the world I mean it's very like like almost zero-sum issue a to view the world say like my great apartment isn't great because my neighbor has one - no my great apartment is great because like look at this dishwasher man yeah you just touch the dish and it's washed right and that is great in and of itself if I have the only apartment or if everybody had the apartment I don't care so you have fundamental gratitude the the world first learned of Geo ha George Hotz in August 2007 maybe before then but certainly in August 2007 when you were the first person to unlock carry unlock an iPhone how did you get into hacking what was the first system you discovered vulnerabilities for and broke into so that was really kind of the first thing I had I had a book in in 2006 called grey hat hacking and I guess I realized that if you acquired these sort of powers you could control the world but I didn't really know that much about computers back then I started with electronics the first iPhone hack was physical card work um you had to open it up and pull an address line high and it was because I didn't really know about software exploitation I learned that all in the next few years and I got very good at it but back then I knew about like how men chips are connected to processors and he knew about software and programming he didn't didn't know I'll really see you the view of the world and computers was physical was the most hard work actually if you read the code that I released with that in August 2007 it's atrocious the language was it a C say yes and in a broken sort of state machine SC I didn't know how to program man so how did you learn to program what was your journey cuz I mean we'll talk about it you've live streams from your programming man this is a chaotic beautiful mess how did you arrive at that years and years of practice I interned at Google after the summer after the iPhone unlock and I did a contract for them where I built hardware for for Street View and I wrote a software library to interact with it and it was terrible code and for the first time I got feedback from people who I respected saying you know like don't write code like this now of course just getting that feedback is not enough the way that I really got good was I wanted to write this thing like that could emulate and then visualize like armed binaries because I wanted to hack the iPhone better and I didn't like that I couldn't like see what that I couldn't single step through the processor because I had no debugger on there especially for the low level things like the boot ROM in the bootloader so I tried to build this tool to do it and I built the tool once and it was terrible I built the tool second times it was terrible I built the tool third time this by the time I was at Facebook it was kind of okay and then I built the tool fourth time when I was a Google intern again in 2014 and that was the first time I was like this is finally usable how do you pronounce this kira-kira yeah so it's essentially the most efficient way to visualize the change of state of the computer as the program is running that's what I mean by debugger yeah it's a timeless debugger so you can rewind just as easily as going forward think about if you're using gdb you have to put a watch on a variable if you want to see if that variable changes and Kure you can just click on that variable and then it shows every single time when that variable was changed or accessed think about it like get for your computers uh the run lock so there's like a deep log of of the state of the computer as the program runs and you can rewind why isn't that maybe it is maybe you can educate me what isn't that kind of debugging used more often ah because the tooling is bad well two things one if you're trying to debug chrome chrome is a 200 megabyte binary that runs slowly on desktops so that's going to be really hard to use for that but it's really good to use for like CTFs and for boot roms and for small parts of code so it's it's hard if you're trying to debug like massive systems what's the CTF and what's the boot ROM the boot ROM is the first code that executes it's the minute you give power to your iPhone okay and CTF were these competitions that I played capture the flag to capture the flag I was going to ask you about that what are those LaVette I watched a couple videos on YouTube those look fascinating what have you learned about maybe at the high level of vulnerability of systems from these competitions the like I feel like like in the heyday of CTFs you had all of the best security people in the world challenging each other and coming up with new toy exploitable things over here and then everybody okay who can break it and when you break it you get like there's like a file on the server called flag and then there's a program running listening on a socket that's vulnerable so you write an exploit you she'll and then you cat flag and then you type the flag into like a web-based scoreboard and you get points so the goal is essentially to find an exploit in the system that allows you to run shell to run arbitrary code on that system that's one of the categories that's like the PO noble category vulnerable yeah horrible it's like you know you pwned the program you are it's a program yeah yeah you know for personally I apologize I'm gonna I'm gonna say it's because I'm Russian but maybe you can help educate me some video game like misspell to own way back in the Mia and there's just I wonder if there's a definition I'll have to go to urban dictionary for it okay so what was the heyday seat yeah by the way but was it what decade are we talking about I think like I mean maybe I'm biased because it's the era that that that I played but like 2011 to 2015 because the modern CTF scene is similar to the modern competitive programming scene you have people who like do drills you have people who practice and then once you've done that you've turned it lesson to a game of generic computer skill and more into a game of okay you memorize you you drill on these five categories and then before that it wasn't it didn't have like as much attention as it had I don't know they were like I won $30,000 ones in Korea for one of these competitions oh crap they were they were that so that means I mean money's money but that means there was probably good people there exactly yeah are the challenges human constructive or are they grounded in some real flaws and real systems usually they're human constructed but they're usually inspired by real flaws what kind of systems are imagined is really focused on mobile like what has vulnerabilities these days is it does primarily mobile systems like Android everything does No yeah of course the price has kind of gone up because less and less people can find them and what's happened in security is now if you want to like jailbreak an iPhone you don't need one exploit anymore you need nine nine chained together what women yeah Wow okay so it's really so what's the but what's the benefit speaking higher level philosophically about hacking I mean it sounds from everything I've seen about you you just love the challenge and you don't want to do anything you don't want to bring that exploit out into the world and doing the actual let it run wild you just want to solve it and then you go on to the next thing oh yeah I mean doing criminal stuffs not really worth it and I'll actually use the same argument for why I don't do defense for why I don't do crime if you want to defend a system say the system has ten holes right if you find nine of those holes as a defender you still lose because the attacker gets in through the last one if you're an attacker you only have to find one out of the ten but if you're a criminal if you log on with a VPN nine out of the ten times but one time you forget you're done because you're caught okay because you only have to mess up once to be caught as a criminal yeah that's why I'm not a criminal but okay let me uh that's having a discussion with somebody just at a high level about nuclear weapons actually why we're having blowing ourselves up yet and my feeling is all the smart people in the world look at the distribution of smart people smart people are generally good and then this other person I was talking to Sean Carroll the physicist and you were saying no good and bad people are evenly distributed amongst everybody my sense was good hackers are in general good people and they don't want to mess with the world what's your sense I'm not even sure about that like I have a nice life crime wouldn't get me anything but if you're good and you have these skills you probably have a nice life too right like you can use the father things but is there an ethical is there some is there a little voice in your head that says well yeah if you could hack something to where you could hurt people and you could earn a lot of money doing it though not hurt physically perhaps but disrupt her life in some kind of way it is there a little voice that says um what two things one I don't really care about money so like the money wouldn't be an incentive the thrill might be an incentive but when I was 19 I read crime and punishment right that was another that was another great one that talked me out of ever really doing crime Oh cuz it's like that's gonna be me I'd get away with it whatever just went in my head even if I got away with it you know and then you do crime for long enough you'll never get away with it that's right in the end that's a good reason to be good I wouldn't say good I just say I'm not bad you're a talented programmer and a hacker in a good positive sense of the word award you've played around found vulnerabilities in various systems what have you learned broadly about the design of systems and so on from that from that whole process you learn to not take things for what people say they are but you look at things for what they actually are yeah I understand that's what you tell me it is but what does it do man and you have nice visualization tools to really know what it's really doing oh I wish I'm a better programmer now than I was in 2014 I said Kira that was the first tool that I wrote that was usable I wouldn't say the code was great I still wouldn't say my code is great so how was your evolution as a programmer except practice he went he started with C at which point did you pick up Python because you're pretty big and Python though now yeah in uh in college I went to Carnegie Mellon when I was 22 um I went back I'm like I'm gonna take all your hardest CS courses we'll see how I do right like did I miss anything by not having a real undergraduate education took operating systems compilers AI and they're like a freshman reader math course and operating says some of these some of those classes you mentioned actually they're great at least one the 2012 circuit 2012 operating systems and compilers we're two of the best classes I've ever taken my life because you write an operating system and you write a compiler I wrote my operating system in C and I wrote my compiler in Haskell but classical well somehow I picked up Python that semester as well I started using it for the CTS actually that's when I really started to get into CTF and CTF you're all to race against the clock so I can't write things and say oh there's a clock component so you really want to use the programming language you can be fastest than 48 hours pone as many of these challenges you can pone yeah you got like a hundred points a challenge whatever team gets the most you were both the Facebook and Google for a brief stint yeah well the project zero actually at Google for five months where you develop kara what was project zero about in general speak what what just curious about the security efforts in these companies well product zero started the same time I I went there what what years are there 2015 2015 so that was right at the beginning of project it's small it's Google's offensive security team I'll try to give I'll try to give the best public facing explanation that I can so the idea is basically these vulnerabilities exist in the world nation states have them some high powered bad actors have them sometime people will find these vulnerabilities and submit them in bug bounties to the companies but a lot of the companies don't really care it only fix the bug there's no it doesn't hurt for there to be a vulnerability so project zero is like we're gonna do it different we're going to announce a vulnerability and we're going to give them 90 days to fix it and then whether they fix it or not we're gonna drop the drop the zero day oh wow we're gonna drop the weapon that's so cool that is so cool I love that deadlines though that's so cool give him real deadlines yeah and I think it's done a lot for moving the industry forward I watched your coding sessions on the stream downline you code things up basic projects usually from scratch I would say sort of as a programmer myself just watching you that you type really fast and your brain works in both brilliant and chaotic ways I don't know if that's always true but certainly for the live streams so it's it's interesting to me because I'm more I'm much slower and systematic and careful and you just move I mean probably an order of magnitude faster some curious is there a method to your madness is this just who you are there's pros and cons there's pros and cons to my programming style and I'm aware of them like if you ask me to like like get something up and working quickly with like an API that's kind of undocumented I will do this super fast because I will throw things at it until it works if you ask me to take a vector and rotate it 90 degrees and then flip it over the XY plane I'll spam program for two hours and won't get it all because it's something that you could do with a sheet of paper think through design and then just you really just throw stuff at the wall and you get so good at it that it usually works I should become better at the other kind as well sometimes I'll do things pathetically it's nowhere near as entertaining on the twitch streams I do exaggerate it a bit on the edge games as well the twitch streams I mean what do you want to see a game or you want to see actions permit me right I'll show you a PM for programming yes I recommend people go to I think I watched I was probably several hours you put like I've actually left you programming in the background while I was programming because you made me you it was it was like watching a really good gamer it's like energizes you because you're like moving so fast it so it's it's awesome it's inspiring and so it made me jealous that like because my own program is inadequate in terms of speed Oh as I was like so I'm twice as frantic on the live streams as I am when I code without oh it's super entertaining so I I wasn't even paying attention to where you were coding which is great it's just watching you switch windows and VAM I guess is driven screen I've developed a workflow Facebook and talk about how do you learn new programming tools ideas techniques these days what's your like methodology for learning new things so I wrote for comma the distributed file systems out in the world are extremely complex like if you want to install something like like like Saif Saif is I think the like open infrastructure to should be a file system or there's like newer ones like seaweed FS but these are all like 10,000 plus line projects I think some of them are even 100,000 line and just configuring them as a nightmare so I wrote I wrote one um it's 200 lines and it's it uses like nginx to the live servers and has low master server that I wrote and go and the way I go this if I would say that I'm proud per line of any code I wrote maybe there's some exploits that I think are beautiful and then this this is 200 lines and just the way that I thought about it I think was very good and the reason it's very good is because that was the fourth version of it that I wrote and I had three versions that I threw away you mentioned you see go I ready go yeah and go so is that a functional language I forget what goes they go is Google's language right I'm a functional it's some it's like in a way it's C++ but easier it's it's strongly typed it has a nice ecosystem erotic when I first looked at it I was like this is like Python but it takes twice as long to do anything yeah now that I've open pilot is migrating to sea but it still has large Python components I now understand why Python doesn't work for large code bases and why you want something like Oh interesting so why why doesn't Python work for so even most speaking for myself at least like we do a lot of stuff basically demo level work with autonomous vehicles and most of the work is Python yeah why doesn't Python work for large code bases because well lack of type checking is a big errors creeping yeah and like you don't know the compiler can tell you like nothing right so everything is either you know like like syntax errors fine but if you misspell a variable and Python the compiler won't catch that there's like linters that can catch it some other time there's no types this is really the biggest downside and then will Python slow but that's not related to it well maybe the kind of related to its that's lacking so what's what's in your toolbox these days is a Python what else go I need to move on something else but my adventure interdependently type languages I love these languages they just have like syntax from the 80s what do you think about JavaScript yes thanks Nick tomorrow typescript javascript is the whole ecosystem is unbelievably confusing NPM updates a package from zero to two to zero to five and that breaks your babble linter which translates your es5 into es6 which doesn't run on so why do I have to compile my JavaScript again huh it may be the future though if you think about I mean I've embraced JavaScript recently because just like I've continually embraced PHP it seems that these worst possible languages live on for long is that cockroaches never die yeah well it's in the browser and it's fast it's fast yeah it's in the browser and compute mites they become you know the browser it's unclear what the role the browser's in terms of distributed computation in the future so javascript is definitely here to stay yeah interesting if Tom's vehicles will run on JavaScript one day I mean you have to consider these possibilities well all our debug tools are JavaScript we actually just open-source them we have a tool Explorer which you can annotate your dis engagements and we have tool cabana which lets you analyze the canned traffic from the car so basically any time you're visualizing something about the log you using javascript yeah well the web is the best UI toolkit by far yeah um so and then you know what you're voting in JavaScript we have a react guy he's good he acts nice let's get into it so let's talk to Thomas vehicles you found it comma a let's at a high level how did you get into the world the vehicle automation can you also just for people who don't know tell the story of comma yeah sure so I was working at this AI startup and a friend approached me and he's like dude I don't know where this is going but the coolest applied AI problem today is self-driving cars I'm like well absolutely do you want to meet with UI mosque and he's looking for somebody to build a vision system for auto pilot this is when they were still on ap one they were still using mobile I kneel on back then was looking for a replacement and he brought me in and we talked about a contract where I would deliver something that meets mobile eye level performance I would get paid twelve million dollars if I could deliver it tomorrow and I would lose 1 million dollars for every month I didn't deliver yeah so I was like ok this is a great deal this is a super exciting challenge you know what even if it takes me 10 months I get two million dollars it's good maybe I can finish up in five maybe I don't finish it at all and I get paid nothing and I'll work for twelve months for free so maybe I just take a pause on that I'm also curious about this because I've been working on robotics for a long time and I'm curious to see a person like you just step in and sort of somewhat naive but brilliant right so that's though that's the best place to be because you basically full-steam take on a problem how confident how from that time because you know a lot more now at that time how hard do you think it is to solve all of autonomous driving I remember I suggested to Elon in the meeting I'm putting GPU behind each camera to keep the compute local this is an incredibly stupid idea I leave the meeting 10 minutes later and I'm like I could have spent a little bit of time thinking about this problem was I would just send all your cameras to one big GPU you're much better off doing that oh sorry you said behind every camera you have a small GPU I was like oh I'll put the first few layers of my comm there Oh like why did I say that that's possible it's possible but it's a bad idea it's not obviously a bad idea pretty obvious but whether it's actually a bad idea or not I left that meeting with Elon like beating myself up I'm like why did I say something stupid yeah you haven't given I'm at least like thought through every aspect yes he's very sharp too like usually in life I get away with saying stupid things and then kind of course alright right away he called me out about it and like usually in life I get away with saying stupid things and then like people will you know people a lot of times people don't even notice and I'll like correct it and bring the conversation back but with Elon it was like nope like okay well that's not at all why the contract fell through I was much more prepared the second time I met him yeah but in general huh how hard did you think it is like 12 months is uh-oh is it tough timeline oh I just thought I'd clone mob like you three I didn't think I'd solve level five self-driving or anything so the goal there was to do lane-keeping good good link keeping I saw my friend showed me the outputs from a mobile I in the office from a mobile I was just basically two lanes at a position of a lead car mm-hm like I can I can gather a dataset and train this net in in weeks and I did well first time I tried the implementation of mobile I and the test I was really surprised how good it is it's quite incredibly good because I thought it's just because I've done a lot of computation I thought it'd be a lot harder to create a system that that's stable so I was personally surprised you know have to admit it because I was kind of skeptical before trying it because I thought it would go in and out a lot more it would get disengaged a lot more and it's pretty robust so what how how hard is the problem we need to when you tackled it I think a p1 was great like Elon talked about dis engagements on the 405 down in LA we'd like the lane marks were kind of faded and the mobile eye system would drop out uh like I had something up and working that I would say was like the same quality in three months same quality but how do you know you you say stuff like that yeah confidently but you can't and I love it but well the question is you can't you're kind of going by feel because he not solely absolutely like like I would take I hadn't I borrowed my friends Tesla yeah I would take ap one out for a drive yeah and then I would take my system out for a dry and seems reasonably like the same so the four or five how hard is it to create something that could actually be a product that's deployed I mean I've read an article or you on this respondent said something by you saying that to build autopilot is is more complicated than a single George Hotz a level job how hard is that job to create something that would work across the globe Lee what are the global ease the challenge but Elon followed that up by saying it's gonna take two years in a company of ten people yeah and Here I am four years later with a company of twelve people and I think we still have another two to go two years so yeah so what do you think what do you think about the hottest is progressing with autopilot v2 v3 I think we've kept pace with them pretty well I think navigator autopilot is terrible we had some demo features internally of the same stuff and we would test it and I'm like I'm not shipping this even as like open-source software to people what do you think is do Consumer Reports does a great job of describing it like when it makes a lane change it does it worse than a human you shouldn't ship things like autopilot open pilot they Lane keep better than a human if you turn it on for a stretch of highway like an hour long it's never gonna touch a lane line human will touch probably a lane line twice you just inspired me I don't know if you're grounded and data on that I read labor okay but no but that's interesting uh I wonder actually how often we touch Lane lines in general like a little bit cuz it is okay I could answer that question pretty easily with the common data side yeah I'm curious I've never answered it I don't know yeah I just - is like my person it feels right that's interesting because every time you touch the lane that's the source of a little bit of stress and kind of lane-keeping is removing that stress that's all to me the big the biggest value-add honestly is just removing the stress of having to stay in lane and I think honestly I don't think people fully realize first of all that that's a big value add but also that that's all it is and that not only I find it a huge value add I drove down when we moved to San Diego I drove down our Enterprise rent-a-car and I missed it so I missed having the system so much it's so much more tiring to drive without it it's it is that Lane centering that's the key feature yeah and in a way it's the only feature that actually adds value to people's lives and autonomous vehicles today way mode does not add value to people's lives it's a more expensive lower slower uber maybe someday it'll be this big cliff where it adds value but I don't usually do this vessei I haven't talked to is that this is good because I haven't I have intuitively but I think we're making it explicit now I I actually believe that really good lane-keeping is a reason to buy a car will be a reason to buy a car is a huge value add I've never until we just started talking about it haven't really quite realized that that I've felt with elan chase of level four is not the correct chase it was on because you should just say Tesla has the best as if from a testing perspective say Tesla has the best lane-keeping coming I should say coming I is the best link keeping and that is it yeah yeah does do you think well you have to do the longitudinal as well you can't just Lane keep you have to do a cc but a cc is much more forgiving than lanky especially on the highway oh by the way are you uh calming eyes camera only correct oh no we use the radar we from the car you were able to get to open it um we can't do a camera only now it's gotten to the point but we leave the radar there is like a it's it's fusion now okay so let's maybe talk through some of the system specs on the hardware or what it what's what's the hardware side of what you're providing what's the capabilities in the software side would open pilot and so on so open pilot as the the box that we sell that it runs on it's a phone in a plastic case it's nothing special we sell it without the software so you're like you know you buy the phone it's just easy it'll be easy setup but it's sold with no software open pilot right now is about to be 0.6 when it gets to 1.0 I think we'll be ready for a consumer product we're not gonna add any new features we're just gonna make the lane-keeping really really good so what do we have right now it's a snapdragon 820 say so many IMX 298 forward-facing camera driver monitoring camera and she's a selfie cam on the phone and a can transceiver biffle's little thing calls pandas and they talk over USB to the phone and then they have three canvases that they talk to the car one of those campuses is the radar CANbus one of them is the main car CANbus and the other one is the proxy camera CANbus we leave the existing camera in place so we don't turn a DB off right now we still turn a TV off if you're using our longitudinal but we're gonna fix that before 1.0 you got it wow that's cool so in its can both way so how are you able to control vehicles so we proxy the vehicles that we work with already have Lane Keeping Assist system so Lane Keeping Assist can mean a huge variety of things it can mean it will apply a small torque to the wheel after you've already crossed a lane line by a foot which is the system in the older Toyotas versus like I think Tesla still calls it Lane Keeping Assist where it'll keep you perfectly in the center of the lane on the highway you can control like you would in joystick the cars these so these cars already have the capability of drive-by-wire so is it is it trivial to convert a car that it operates with it open pile is able to control the steering Oh a new car or a car that we so we have support now for 45 different makes of cars what are one of the cars general mostly Hondas and Toyotas we support almost every Honda and Toyota made this year and then a bunch of GM's bunch of Subarus which it doesn't have to be like a Prius it could be Coral as well okay the 2020 Corolla is the best car with open pilot it just came out there the actuator has less lag than the older Corolla I think I started watching video with your eye the way you make videos is awesome literally the dealerships streaming stream for an hour yeah and basically like if stuff goes a little wrong you're just like you just go with it yeah I love it what's real yeah that's real that's that's it's that's so beautiful and it's so in contrast to the way other companies would put together a video like that how do I like to do it like good I mean if you become super rich one day is successful I hope you keep it that way because I think that's actually what people love that kind of genuine oh it's all that has value to me yeah my money has no if I sell out to like make money and I sold out it doesn't matter what do I get yacht I don't I got and I think Tesla's actually has a small inkling of that as well with autonomy day they did reveal more than I mean of course there's marketing communications you can tell but it's more than most companies will reveal which is I hope they go towards a direction more other companies GM Ford oh Jessa Tesla's gonna win level 5 they really are so let's talk about it you think you're focused on level 2 currently currently we're gonna be one to two years behind Tesla getting to level five okay we're interested right we're into it you're in I'm just saying once Tesla gets it we're one to two years behind I'm not making any timeline on when Tesla's that's right you did that's brilliant I'm sorry Tesla investors if you think you're gonna have an autonomous robot taxi fleet by the end of the year yes that's all bet against that so that what do you think about this the most level four companies are kind of just doing their usual safety driver during full autonomy kind of testing and then Tesla does basically trying to go from lane-keeping to full autonomy what do you think about that approach how successful would it be a ton better approach because Tesla is gathering data on a scale that none of them are they're putting real users behind the behind the wheel of the car it's I think the only strategy that works the incremental well so there's a few components to test approach that's that's more than just incrementally you spoke with is the one is the software so over-the-air software updates necessity I mean way more ease have those - those aren't but there was differentiating from the automaker's right no link keeping assist systems have no cars with lane keeping system have that except Tesla yeah and the other one is the data the other direction which is the ability to query the data I don't think they're actually collecting as much days people think but the ability to turn on collection and turn it off so I'm both in the robotics world in the the psychology human factors world many people believe that level to autonomy is problematic because of the human factor like the more the task is automated the more there's a vigilance decrement you start to fall asleep you start to become complacent start texting more and so on do you worry about that because if we're talking about transition from lane-keeping to full autonomy if you're spending eighty percent of the time not supervising machine do you worry about what that means to the safety of the drivers one we don't consider open pilot to be 1.0 until we have 100% driver monitoring you you can cheat right now our driver monitoring system there's a few ways to cheat it there pretty obvious we're working on making that better before we ship a consumer product that can drive cars I want to make sure that I have driver monitoring that you can't cheat what's like a successful driver monitoring system look like it's keep its is it all buzz just keeping your eyes on the road um well a few things so that's what we went with it first for driver monitoring I'm checking I'm actually looking at where your head is looking but cameras know about my resolution eyes are a little bit hard to get well head is this big I mean that is good and actually a lot of it just as psychology wise to have that monitor constantly there it reminds you that you have to be paying attention but we want to go further we just hired someone full-time to come onto the driver monitoring I want to detect phone in frame and I want to make sure you're not sleeping how much does the camera see of the body this one not enough not enough the next one everything what's interesting fish Atkins we have we're doing just data collection that real-time but fish eye is a beautiful mouth being able to capture the body and the smartphone is really like the biggest problem I'll show you I can show you one of the pictures from from our finder system awesome so you're basically saying the driver monitoring will be the answer to that um I think the other point that the original paper is is good as well you're not asking a human to supervise a machine without giving them meat they can take over at a time right our safety model you can take over we disengage on both the gas or the brake we don't disengage on steering I don't feel you have to but we disengage on gas or brake so it's very easy for you to take over and it's very easy for you to re-engage that switching should be super cheap yeah the cars that require even autopilot requires a double press that's almost I said I like that yeah and then then the cancel um to cancel in autopilot you either have to press cancel which no one knows where that is so they press the brake but a lot of things you don't you want to press the brake you want present ass so you should cancel on gas or wiggle the steering wheel which is bad as well wow that's brilliant I haven't heard anyone articulate at that point I like what this is all I think about it's because I think I think actually Tesla has done a better job than most automakers at making that frictionless but you just described that it could be even better I love super cruise as an experience once it's engaged yeah I don't know if you've used it but getting the thing to try to engage him yeah I've used this of Germany's super cruise a lot so what's their thoughts on the super Cruiser system in June disengage super cruise and it falls back to ACC so my car's like still accelerating it feels weird otherwise when you actually have super cruise engaged on the highway it is phenomenal we bought that Cadillac we just sold it but we bought it just to like experience this and I wanted everyone in the office to be like this is what we're striving to build GM pioneering with the driver monitoring you know you like their driver monitoring system it has some bugs if there's a sun shining back year it'll be blind to you by overall mostly yeah that's so cool you know the stuff that's uh I don't often talk to people that because it's such a rare car unfortunately they bought one yes possibly for us we lost like by 25k the deprecation but a Philips worth it I was very pleasantly surprised that GM system was so innovative and really that wasn't advertised much wasn't talked about much yeah and I was nervous that it would die that they would disappear my eyes did they put it on the wrong car they should've put it on the bolt and not some weird Cadillac that nobody bought I think that's gonna be into they're saying at least is going to be into their entire fleet so what do you think about it if as long as we're on the driver monitoring what do you think about you know I must claim that driver monitoring is not needed normally I love his claims that one is stupid that one is stupid and you know he's not gonna have his level five fleet by the end of the year hopefully he's like okay I was wrong I'm gonna add driver monitoring because when these systems get to the point that they're only messing up once every thousand miles you absolutely need driver monitor so let me play Delta because I agree with you but let me play devil's advocate so one possibility is that without driver monitoring people are able to monitor the self-regulate monitor themselves you know that so your idea is seeing all the people sleeping in decimals uh yeah well I'm a little skeptical of all the people sleeping in Tesla's because I have I've stopped paying attention to that kind of stuff because I want to see real data there's too much glorified it doesn't feel scientific to me so I want to know you know what how many people are really sleeping in Tesla's vs. sleeping I've I was driving here sleep-deprived in a car with no automation I was falling asleep I agree that it's high P it's just like you know what if you under I've am wondering I think I rented a my last autopilot experience was I rented a model three in march and drove it around the wheel thing is annoying and the reason the wheel thing is annoying we use the wheel thing as well but we don't disengage on wheel for Tesla you have to touch the wheel just enough you should trigger the torque sensor to tell it that you're there but not enough as to disengage it which don't use it for two things you disengage one wheel you don't have to that whole experience Wow beautiful put that all those elements even if you don't have driver monitoring that whole experience needs to be better driver monitoring I think would make I mean I think super cruise is a better experience once it's engaged over autopilot I think super cruise is our transition to engagement and disengagement are significantly worse yeah so there's a tricky thing because if I were to criticize super cruise is uh it's a little too crude and uh I think it's like six seconds or something if you look off-road you'll start warning you it's some ridiculously long period of time and just the way it I think it's basically it's a binary chili adapter it yeah it's it just needs to learn more about you and used to communicate what it sees about you more like I'm not you know Tesla shows what it sees about the external world it would be nice the supercruise would tell us what it sees about the internal world it's even worse than that you press the button to engage and it just says super cruise unavailable yeah why why yeah that transparency is good we've renamed the driver monitoring packet to driver state service state we have car state packet which has the state of the car driver state packet which I stay the driver so what does itah make their BAC must be do you think that's possible with computer vision absolutely so to me it's an open question I don't haven't looked into too much they actually had quite seriously looked at the literature it's not obvious to me that from the eyes and so on you can tell you might need to stuff from the car as well yeah you might need how they're controlling the car right and that's fundamentally at the end of the day what you care about you but I think especially when people are really drunk they're not controlling the car nearly smoothly as they would look at them walking right there the car is like an extension of the body so I think you could totally detect and if you could fix people who drunk distracted asleep if you fix those three yeah this is that's huge so what are the current limitations of open pilot what are the main problems that still need to be solved um we're hopefully fixing a few of them in 0-6 we're not as good as auto pilot at stop cars so if you're coming up to a red light at like 55 so it's the radar stopped car problem which is responsible to auto pilot accidents it's hard to differentiate a stopped car from a like signpost yes that ecology um so you have to fuse you have to do this visually there's no way from the radar data to tell the difference maybe you could make a map but I really believe in mapping at all anymore um really what you don't believe in mapping no so you basically the open pilot solution is saying react to the environment is just like human doing beings and then eventually when you want to do navigate on open pilot I'll train the net to look at ways all runways in the background I'll train a car using GPS at all we use it to crown trees we use it to very carefully ground treat the paths we have a stack which can recover a relative to 10 centimeters over one minute and then we use that to ground truth exactly where the car went in that local part of the environment but it's all local how are you testing in general just for yourself like experiments stuff all right were you were you located San Diego San Diego yeah okay Oh what you basically drive around there then collect some data and watch on Florence we have a simulator now and we have our simulators really cool our simulator is not it's not like a unity based simulator our simulator lets us load in real estate what I mean we can load in a drive and simulate what the system would have done on the historical data ooh nice interesting so what yeah right now we're only using it for testing but as soon as we start using it for training what's your feeling about the real world versus simulation do you like simulation for training if this moves to training Chuck we have to distinguish two types of simulators right there's a simulator that light is completely fake I could get my car to drive around in GTA mm-hmm um I feel that this kind of simulator is useless you're never there's so many my analogy here is like okay fine you're not solving the computer vision problem but you're solving the computer graphics problem right and you don't think you can get very far about creating ultra realistic graphics no because you can create ultra realistic graphics of the road now create alter a realistic behavioral models of the other cars oh well I'll just use my self-driving no you won't you need real you need actual human behavior because that's what you're trying to learn the dead driving does not have a spec the definition of driving is what humans do when they drive whatever way mode does I don't think it's driving right well I think if you win more than others its if there's any useful reinforcement learning I've seen it used quite well I study pedestrians a lot too is try to train models from real data of how pedestrians move and try to use reinforcement learning models to make pedestrians move in human-like ways by that point you've already gone so many layers you detected a pedestrian did you did you hand code the feature vector of their state did you guys learn anything from computer vision before deep learning well okay you know I feel like this is a perception to you is the sticking point does that mean what what's what's the hardest part of the stack here there is no human understandable feature vector separating perception and planning that's the best way I can I can put that there is no so it's all together and it's it's a that's a joint problem so you can take localization localization and planning there is a human understandable feature vector between these two things I mean okay so I have like three degrees position three degrees orientation and those derivatives maybe those second derivatives right that's human understandable that's physical the between perception and planning um so like way Moe has a perception stack and then a planner um and one of the things way matters right is they have a simulator that can separate those two they can like replay their perception data and test their system which is what I'm talking about about like the two different kinds of simulators there's the kind that can work on real data and is the kind of can't work on real data now the problem is that I don't think you can hand code a feature vector right like like you have some lists of like well here's my list of cars on the scenes here's my list of pedestrians in the scene this isn't what humans are doing what are humans doing global some something you're saying that's too difficult to handle I'm saying that there is no state vector given a perfect I could give you the best team of engineers in the world to build a perception system and the best team to build a planner all you have to do is define the state vector that separates those two I'm missing the state vector that separates those two what do you mean so what is the output of your perception system I'll put it the perception system it's theirs okay well there's several ways to do it one is this lamp components localization the other is drivable area drivable space drivable space and then there's the different objects in the scene and different objects in the scene over time maybe to give you input to then try to start modeling the trajectories of those objects sure that's it I can give you a concrete example of something you missed what's that so say there's a bush in the scene humans understand that when they see this bush that there may or may not be a car behind that bush drivable area and a list of objects does not include that humans are doing this constantly at the simplest intersections so now you have to talk about occluded area right right but even that what do you mean by occluded okay so I can't see it well if it's the other side of a house I don't care what's the likelihood that there's a car in that occluded area right and if you say okay we'll add that I can come up with 10 more examples that you can't add certainly occluded area would be something that simulator would have because it's simulating the entire you know occlusion is part of it a part of a vision stack pleasures that what I'm saying is if you have a hand engineered if your perception system output can be written in a spec document it is incomplete yeah idem you know certainly it's it's hard to argue with that because in the end that's going to be true yes I'll tell you what the output of our perception system is was that it's a thousand it's a thousand twenty four dimensional vector training underling oh no not it's a thousand twenty four dimensions of who knows what because its operating on real data yeah yeah and that's the perception that's the perception stake right think about a think about an autoencoder four phases alright if you have an autoencoder four phases and you say it has 256 dimensions in middle and I'm taking a face over here and projecting it to a face over here yeah can you hand label all 256 of those dimensions well no but those are generated automatically but they but even if you tried to do it by hand could you come up with a spec for your and between your encoder and your decoder no no because that's not it is it wasn't designed but there no no but if you could design it if you could design a face Reconstructor system could you come up with a spec no but I think we're missing here a little bit I think the the you're just being very poetic about expressing a fundamental problem of simulators that they're going to be missing so much that the feature vector would just look fundamentally different from in the simulated world in the real world I'm not making a claim about simulators I'm making a claim about the spec division between perception and planning and planning even in your system just in general right just in general if you're trying to build a car that drives if you're trying to hand code the output of your perception system like saying like here's a list of all the cars in the scene here's a list of all the people here's a list of the included areas here's a vector of drivable areas insufficient and if you start to believe that you realize that what Wayman crews are doing is impossible currently what we're doing is the perception problem it's converting the scene into a chessboard you yeah and then you reason some basic reasoning around that chessboard yeah and you're saying that really there's a lot missing there first of all why are we talking about this cuz isn't this a full autonomy is this something you think about oh I want to win self-driving cars so you're really your definition of win includes level of fool five level five I don't think level four is a real thing I want to build I want to build the alphago of driving so so alphago is really end to end yeah is uh yeah it's end to end and do you think this whole problem is those that also kind of what you're getting at with the perception and the planning is that this whole problem the right way to do it is really to learn the entire thing I'll argue that not only is it the right way it's the only way that's going to exceed human performance well certainly true for go everyone who tried to hand code go things built human inferior things and then someone came along and wrote some 10,000 line thing that doesn't know anything about go that beat everybody it's 10,000 lines true in that sense the the open question then that maybe I can ask you is uh driving is much harder than go the open question is how much harder so how because I think the AH mosque approach here with planning and perception it's similar to what you're describing which is really turning into not some kind of modular thing but really do formulate is a learning problem and it solves a learning problem of scale so how many years put one is how many years would it take to solve this problem or just how hard is this freaking problem well the cool thing is I think there's a lot of value that we can deliver along the way I think that you can build lane-keeping assist actually plus adaptive cruise control plus okay looking at ways extends to like all of driving yeah most of driving varies oh your adaptive cruise control treats red lights like cars okay so let's jump around with you you mentioned that you didn't like navigate an autopilot yeah what advice how would you make it better do you think as a feature that if it's done really well it's a good feature I think that it's too reliant on like hand coded hacks for like how does navigate an autopilot do a lane change it actually does the same lane change every time and it feels mechanical humans do different lane changes human sometime will do a slow one sometimes do a fast one navigate an autopilot at least every time I used it it did the identical language how do you learn I mean this is a fundamental thing actually yeah is uh the braking and an accelerating something that's still test the probably does it better than most cars but it still doesn't do a great job of creating a comfortable natural experience and navigate on autopilot just lane changes an extension of that so how do you learn to do natural lane change so we have it and I can talk about how it works so I feel that we have the solution for lateral but we don't yet have the solution for longitudinal there's a few reasons longitudinal is harder than lateral the lane change component the way that we train on it very simply is like our model has an input for whether it's doing a lane change or not and then when we train the end-to-end model we hand label all the lane changes because you have to I struggled a long time about not wanting to do that but I think you have to because you order the training data for the train data right well we actually we have an automatic ground truth or which automatically labels all the lane changes was that possible to automatically label interest yeah and detect the lane I see when it crosses it right I don't have to get that that high percent accuracy but it's like 95 good enough now I set the bit when it's doing the lane change in the end-to-end learning and then I set it to zero when it's not doing a lane change so now if I wanted to do a lane change a test time I just put the bit to a 1 and I'll do later yeah but so if you look at the space of lane change you know some percentage not a hundred percent that we make as humans is not a pleasant experience because we messed some part of it up yeah it's nerve-racking to change even look at the seizure des accelerate how do we label the ones that are natural and feel good you know that's the because that's your ultimate criticism the current Oh navigate not apologies doesn't feel good well the current navigator on autopilot is a hand coded policy written by an engineer in a room who probably went out and tested it a few times on the 280 probably a more a better version of that but yes that's how we would have written it a comment yeah Tesla they tested it and it might have been two engineers yeah no but so if you learn the lane change if you learn how to do a lane change from data just like just like you have a label that says lane change and then you put it in when you want to do the lane change it'll automatically do the lane change that's appropriate for the situation now to get it the problem of some humans do bad lane changes we haven't worked too much on this problem yet it's not that much of a problem in practice my theory is that all good drivers are good in the same way and all bad drivers are bad in different ways and we've we've seen some data to back this up well beautifully put so you just basically if that's true yeah hypothesis then you know task is to discover the good drivers um the good drivers stand out because they're in one cluster and the bad drivers are scattered all over the place and your net learns the cluster yeah that's uh so you just learned from the good drivers and they're easy to cluster we learned from all of them and that automatically learns the policy that's like the majority but we'll eventually probably afterthought so if that theory is true I hope it's true because the the counter theory is there is many clusters maybe but rarely many clusters of good drivers because if there's one cluster of good drivers you can at least discover a set of policies you can learn a set of policies which would be good universally yeah that would be a nice that would be nice if it's true and you're saying that there are some evidence that let's say lane changes can be clustered into four clusters right right there's this finite level of I would argue that all four of those are good clusters all the things that are random are noise and probably bad and which one of the four you pick or maybe it's Tanner maybe it's twenty you can learn them it's context dependent it depends on the scene and the hope is it's not too dependent on the driver yeah the hope is that it all washes out the hope is that there's that the distribution is not bimodal the hope is that it's a nice gas man so what advice would you give to Tessa how to fix how to improve navigate an autopilot the lessons you've learned from Kamiya the only real advice I would give to Tesla is please put driver monitoring in your cars with respect to improvement you can't do that anymore I said to interrupt but you know there's a practical nature of many of hundreds of thousands of cars being produced that don't have a good driver facing camera the model 3 has a selfie cam is it not good enough did they not have put IR LEDs for night that's a good question but I do know that the is fisheye in its relatively low resolution so it's really not this I he wasn't it wasn't designed for Arman you can hope that you can kind of scrape up and and and have something from it yeah but put it in today put it in today today every time I've heard Carpathia talk about the problem and talking about life software 2.0 and how the machine learning is gobbling up everything I think this is absolutely the right strategy I think that he didn't write navigate on autopilot I think somebody else did and kind of hacked it on top of that stuff I think what Carpathia says wait a second why did we hand code this lane change policy with all these magic numbers we're gonna learn it from data they'll fix it they already know what to do there well that that's that's Andres job is to turn everything into a learning problem and collect a huge amount of data the the reality is though not every problem could be turned into a learning problem in the short term in the end everything would be a learning problem the reality is like if you want to build alpha vehicles today it will likely involve no learning and that's that's the the reality is so at which point does learning start it's the crutch statement that lidar is a crutch on which point will learning get up to part of human performance it's all over human performance and imagenet classification under ivan is the question still it is a question I'll say this I'm I'm here to play for 10 years I'm not here to try to I'm here to play for 10 years and make money along the way I'm not here to try to promise people that I'm gonna have my l5 taxi Network up and working in two years do you think those mistake yes what do you think there was the motivation behind saying that other companies are also promising alpha vehicles with their different approaches in 2020 2021 2022 if anybody would like to bet me that those things do not pan out I will I will bet you even money even money I'll bet you as much as you want so are you worried about what's going to happen because you're not in full agreement on that I was going to happen when 2022 21 come around and nobody has fleets of autonomous vehicles no you can look at the history if you go back five years ago they were all promised by 2018 and 2017 but they weren't that strong of promises I mean Ford really declared pretty that I think not many have declared as as like definitively as they have now these dates well okay so let's separate l4 and l5 do I think that it's possible for way mo to continue to kind of like like hack on their system until it gets to level 4 in Chandler Arizona yes knows no safety driver Chandler Arizona yeah but by OSI which year are we talking about oh I even think that's possible by like 2020 2021 but level 4 Chandler Arizona not level 5 New York City level 4 meaning some very defined streets it works out really well very defined streets and then these streets are pretty empty if most of the streets are covered in way MOS we mo can kind of change the definition of what driving is hmm right if your self-driving network is the majority of cars in an area they only need to be safe with respect to each other and all the humans will need to learn to adapt to them now go drive in downtown New York oh yeah that's already you can talk about autonomy in like like fun farms it already works great because you can really just follow the GPS line so what does success look like for comm AI what what are the milestones like where you can sit back with some champagne and say we did it boys and girls well it's never over yeah but don't be let's drink champagne everything straight so what is a good what are some wins um a big milestone that we're hoping for by mid next year is profitability of the company and we're gonna have to revisit the idea of selling a consumer product but it's not gonna be like the comma one when we do it it's gonna be perfect open pilot has gotten so much better in the last two years we're gonna have a few a few features we're gonna have a hundred percent driver monitoring we're gonna disable no safety features in the car um actually I think it'd be really cool we're doing right now our project this week is we're analyzing the data set and looking for all the AEP triggers from the manufacturer systems we have a better data set on that than the manufacturers how much does how many does Toyota have ten million miles of real-world driving to know how many times they're AUB triggered so let me give you cuz yes right financial advice yeah cuz I work with a lot of automakers and one possible source of money for you which I'll be excited to see you take on is basically selling the data so which is something that most people are not selling in a way we're here here at automaker but creating we've done this actually at MIT not for money purposes but you could do it for significant money purposes and make the world a better place by creating a consortium where automakers would pay in and then they get to have free access to the data and I I think a lot of people are really hungry for that and would pay significant amount of money for it here's the problem with that I like this idea all in theory he'd be very easy for me to give them access to my servers and we already have all open source tools to access this data it's in a great format we have a great pipeline but they're gonna put me in the room with some business development guy mm-hmm and I'm gonna have to talk to this guy and he's not gonna know most of the words I'm saying I'm not willing to tolerate that okay but I think I agree with you I'm the same way but you just tell them the terms and there's no discussion needed if if I could just tell them the terms yeah and then like all right who wants access to my data I will sell it to you for let's say you want to go on a subscription I'll sell you 400 a month any 100k mo 100k month I'll give you access to the data subscription yeah yeah I think that's kind of fair came up with that number off the top of my head if somebody sends me like a three line email where it's like we would like to pay a hundred K month to get access to your data we would agree to like reasonable privacy terms of the people who are in the data set I would be happy to do it but that's not gonna be the email the email is gonna be hey do you have some time in the next month where we can sit down and we can I don't have time for that we're moving too fast yeah you could politely respond to that email but not saying I don't have any time for your bullshit yeah you say oh well unfortunately these are the terms and so this is we try to we brought the cost down for you in order to minimize the friction of education after here's the whatever it is 1 2 million years dollars a year and you have access and it's not like I get that email from like but okay am I gonna reach out am I gonna hire a business development person who's gonna reach out to the automaker's no way yeah okay if they reached into me I'm not gonna ignore the email I'll come back with something straight yeah if you're willing just pay honeycomb all the facts they don't man I'm happy to to set that up that's what my engineering time but actually quite insightful view you're right yeah probably because many of the automakers are quite a bit of old-school yeah there will be need to reach out and they want it but they they'll need to be some some communication you right mobile eye circuit 2015 had the lowest R&D; spend of any chip maker like purpur and you look at all the people who work for them and it's all business development people because the car companies are impossible to work with yeah so you're you have no patience for that and you're you're legit Android huh I have something to do right like like it's not like it's not like I don't like I don't mean to like be a dick and say like I don't have patience for that but it's like that stuff doesn't help us with our goal of winning self-driving cars if I want money in the short term if I showed off like the actual like the learning tech that we have it's it's somewhat sad like it's years and years ahead of everybody else's not so maybe not Tesla's I think Tesla has similar stuff to us actually yeah I think Tesla's similar stuff but when you compare it to like what the Toyota Research Institute has you're not even close to what we have no comment but I also can't I have to take your comments I ain't into ative Lee believe you but I have to take it with a grain of salt because I mean you you are an inspiration because you basically don't care about a lot of things that other companies care about you don't try to bullshit in a sense like make up stuff so to drive a valuation you're really very real and you're trying to solve the problem and admire that a lot what I don't necessarily fully can't trust you on I do respect it's like how good it is right I can only but I also know how bad others are and so I'll say I'll say two things about don't trust but verify right I'll say two things about that one is try get in a twenty twenty Corolla and try open pal 0.6 when it comes out next month I think already you'll look at this and you'll be like them this is already really good and then I could be doing that all with hand labelers and all with with like like the same approach that like Mobileye uses when we release a model that no law has the lanes in it that only outputs a path mm-hmm then think about how we did that machine learning and then right away when you see and that's gonna be an open pilot that's gonna be an open pilot before 1.0 when you see that model you'll know that everything I'm saying is true because how else did I get that model good one of the things too about the simulator oh yeah yeah this is super exciting that's super exciting and uh but like you know I listened to your talk with Kyle and Kyle was originally building the the after market system and he gave up on it because of technical challenges yeah because of the fact that he's gonna have to support twenty to fifty cars we support forty five because what is he gonna do when the manufacturer ABS system triggers we have alerts and warnings to deal with all of that in all the cars and how is he going to formally verify it well I got ten million miles of data it's probably better it's probably better verified than the spec yeah I'm glad you're here talking to me this is I'll remember this day is this interesting if you look at Kyle's from from Cruz I'm sure they have a large number of business development folks and you work with he's working with GM you could work with agro a I working with Ford it's interesting because chances that you fail business-wise like bankrupt are pretty high yeah and and yet it's the Android model is you're actually taking on the problem so that's really inspiring I mean well I have a long-term way for kamma to make money too and one of the nice things when you really take on the problem which is my hope for autopilot for example is things you don't expect ways to make money or create value that you don't expect will pop up oh I've known how to do it since kind of 2017 is the first time I said it well which part to know it to know how to do which part our long-term plan is to be a car insurance company insurance yeah I love it yeah yeah what I make driving twice is safe not only that I have the best date is that you know who statistically is the safest drivers and oh oh we see you we see you driving unsafely we're not going to insure you and that that causes a like bifurcation in the market because the only people who can't get common insurance or the bad drivers Geico can insure them their premiums crazy higher premiums are crazy low would win contracts take over that whole market okay so if we win if we went but that's I'm saying like how do you turn comma into a ten billion dollar company is that that's right so you you know a musk who else who else is thinking like this and working like this in your view who are the competitors are there people seriously I don't think anyone that I'm aware of as seriously taking on lane-keeping you know like to worse a huge business that turns eventually into full autonomy that then creates yeah like that creates other businesses on top of it and so on thinks insurance thinks all kinds of ideas like that do you know who anyone else thinking like this not really that's interesting I mean it my sense is everybody turns to that in like four or five years like Ford once the autonomy doesn't feel fall through but at this time Elon to the iOS by the way he paved the way for all I was not i OS true I would not be doing comma AI today if it was not for those conversations with Elon and if it were not for him saying like yeah I think he said like well obviously we're not gonna use Leiter we use cameras humans use cameras so what do you think about that how important is lidar everybody else is on l5 is using lidar what are your thoughts on his provocative statement that lidar is a crutch see sometimes we'll say dumb things like the driver monitoring thing but sometimes we'll say absolutely completely 100% obviously true things yeah of course lidar is a crutch it's not even a good crutch you're not even using it they're using it for localization yeah which isn't good in the first place if you have to localize your car to centimetres in order to drive like yeah they're not drive it currently not doing much machine learning I thought polite our data meaning like to help you in the tasks of general tasks of perception the main goal of those light hours on those cars I think is actually localization more than perception or at least that's what they use them for yeah that's true if you want to localize two centimeters you can't use GPS the fanciest GPS in the world can't do it especially if you're under tree cover and stuff flatter I can do it pretty easily see really they're not taking on I mean in some research they're doing they're using it for perception but and they're certainly not which sad they're not fusing it well lay vision they do use it for perception I'm not saying they don't use it for perception but the thing that they have vision based and radar based perception systems as well you could remove the lidar and and and keep around a lot of the dynamic object perception you want to get centimeter accurate localization good luck doing that with anything else so what should Cruz lame-o do like what would you be your advice to them now anyway Mo's actually there's I mean they're doing they're serious way mo out of all of them equate so serious about the long game if everybody fell five is a lot is requires fifty years I think when will be the only one left standing at the end with the forgiving the financial backing if they have Google box um I'll say nice things about both lame-o and Cruz let's do it nice is good way mo is by far the furthest along with technology way mo has a three to five year lead on all the competitors um if that if the way mo looking stack works mm-hmm maybe three year lead if the way mo looking stack works they have a three year lead now I argue that way mo has spent too much money to recapitalize to gain back their losses in those three years also self-driving cars have no network effect like that yeah goober has a network effect you have a market you have drivers and you have riders self-driving cars you have capital and you have riders there's no network effect if I want to blanket a new city in self-driving cars i buy the off-the-shelf Chinese knockoff self-driving cars and I buy enough up from the city I can't do that with drivers and that's why Ober has a first mover advantage that no self-driving car company will can you uh disentangle that a little bit uber you're not talking about uber the autonomous vehicle number you talked about the uber cars okay yeah I'm over I open for business in Austin Texas listen I need to attract both sides of the market I need to both get drivers my platform and riders on my platinum and I need to keep them both sufficiently happy right riders aren't going to use it if it takes more than five minutes for an uber to show up drivers aren't gonna use it if they have to sit around all day and there's no riders so you have to carefully balance a market and whenever you have to carefully balance a market there's a great first mover advantage because there's a switching cost for everybody right the drivers and the riders would have to switch at the same time let's even say that you know um let's say Luber shows up in Luber somehow you know agrees to do things that add a bigger you know you know we're just gonna we've done it more efficiently right Luber is only takes five percent of a cot instead of the ten percent that Hooper takes no one is gonna switch because the switching cost is higher than that five percent so you actually can in markets like that you have a first mover advantage yeah autonomous vehicles of the level five variety have no first mover advantage if the technology becomes commoditized say I want to go to a new city look at the scooters it's gonna look a lot more like scooters every person with a checkbook can blanket a city in scooters and that's why you have 10 different scooter companies yeah which one's gonna win it's a race to the bottom it's terrible market to begin because there's no market for scooters and scooters don't get a say and whether they want to be bought and deployed to a city or not right so the yeah we're gonna entice the scooters with subsidies and deals so whenever you have to invest that capital that's it doesn't it doesn't come back yeah that they can't be your main criticism over the way mo approach oh I'm saying even if it does technically work even if it does technically work that's a problem yeah I don't know I if I were to say I I would I would say you're already there I haven't even thought about that but I would say the bigger challenge is the technical approach so way most cruises and not just the technical approach but of creating value I still don't understand how you beat uber the the human driven cars in terms of financially it doesn't it doesn't make sense to me that people want to want to get an autonomous vehicle I don't understand how you make money in the long term like real long-term but it just feels like there's too much capital investment needed oh and they're gonna be worse than ubers because they're gonna they're gonna stop for every little you know thing everywhere um actually a nice thing about Cruz that was my nice thing about wait another three years that it wasn't nice oh that's three years technically ahead of everybody their tech stack is is great my nice thing about Cruz is GM buying them was a great move for GM for 1 billion dollars GM bought an insurance policy against way mo they put Cruz is three years behind way mo hmm that means Google will get a monopoly on the technology for at most three years and technology works you might not even be right about the three years it might be less might be less crews actually might not be that far behind I don't know how much way mo has waffled around or how much of it actually is just that long tail yeah okay if that's the best you could say there's some nice things it that's more of a nice thing for GM that that's a smart insurance policy it's just more insurance policy I mean I think that's how I I can't see crews working out any other for crews to leapfrog way mo would really surprise me yeah so let's talk about like the underlying assumptions of everything is we're not going to leapfrog Tesla Tesla would have to seriously mess up for us because you're okay so the way you leapfrog right is you come up with an idea or you take a direction perhaps secretly that the other people aren't taking and so cruise way mo even Aurora no Aurora tzuke's is the same stack as well they're all the same codebase even and they're all the same DARPA urban challenge codebase so the question is do you think there's a room for brilliance and innovation there that will change everything like say okay so I'll give you examples it could be if revolution and mapping for example that allow you to map things do HD maps of the whole world all weather conditions really well or revolutionist simulation to where the the what you said before becomes incorrect that kind of thing I knew room for breakthrough innovation um what I said before about oh they actually get the whole thing well I'll say this about we divide driving into three problems and I actually haven't solved the third yet but I have an idea how to do it so there's the static the static driving problem is assuming you are the only car on the road right right and this problem can be solved 100% with mapping and localization this is why farms work the way they do if all you have to deal with is the static problem and you can statically schedule your machines right it's the same as like statically scheduling processes you can statically schedule your tractors to never hit each other on their paths all right because then you know the speed they go at so so that's the static driving problem Maps only helps you with the static driving problem yeah the question about static driving yeah you just made it sound like it's really easy that was really easy how easy how well because the whole drifting out of lane when when Tesla drifts out of lane is failing on the fundamental static driving problem Tesla is drifting out of lane the static driving problem is not easy for the world the static driving problem is easy for one route and one route in one weather condition with one state of lane markings and like no deterioration no cracks in the road I'm assuming you have a perfect localizer so that's all for the weather condition and me the lane marking condition that's the problem is how could you how do you have a perfect you can build perfect localizers are not that hard to build okay come on now with with wood lighter why don't ya wood lighter okay yeah but you use lighter right like use lidar build a perfect localizer building a perfect localizer without lidar it's gonna be it's gonna be hard you can get ten centimeters without liner you can get one centimeter with lidar maybe concern about the one or ten centimeter I'm concerned if every once in a while you're just way off yeah so this is why you have to carefully make sure you're always tracking your position you want to use light camera fusion but you can get the reliability of that system up to a hundred thousand miles and then you write some fallback condition where it's not that bad if you're way off right I think that you can get it to the point it's like özil D that you're you're never in a case where you're way off and you don't know it yeah okay so this is brilliant so that's the static static we can especially with lidar and good HD maps you can solve that problem easy no you just the static static very typical for you to say something's easy I got it it's not as challenging as the other ones okay well it's okay maybe it's obvious how to solve it the third one's the hardest well where do we get and a lot of people don't even think about the third one and even I see it as different from the second one so the second one is dynamic the second one is like say there's an obvious examples like a car stopped at a red light right you can't have that car in your map yeah because you don't know whether that car is gonna be there or not so you have to detect that car in real time and then you have to you know do the appropriate action right also that car is not a fixed object that car may move and you have to predict with that car will dim alright so this is the dynamic problem yeah do you have to deal with this um this involves again like you're gonna need models of other people's behavior do you are you including in that and I don't want to step on on the third one oh but if I are you including in that you're influenced and people I guess the third okay that's the moon we call it the counterfactual yeah I believe that I just talked to Judea pearl who's obsessed with counterfactuals oh yeah yeah so the static and the dynamic yeah our approach right now for lateral will scale completely to the static a dynamic the counterfactual the only way I have to do it yet they don't give you thing that I want to do once we have all these cars is I want to do reinforcement learning on the world I'm always gonna turn the exploiter up to max I'm not gonna have them explore but the only real way to get at the counterfactual is to do reinforcement learning because the other agents are humans so that's fascinating that you break you down like that I agree completely I've set my life thinking about this beautiful they're so and part of it because you're slightly insane because not my life just the last four years no no you have like some some nonzero percent of your brain has a madman in it which that's a really good feature but there's a safety component to it that I think when this sort of counterfactuals and so on that would just freak people out how do you even start to think about just in general I mean you've you've had some friction with Nitza and so on I am frankly exhausted by safety engineers the the prioritization on safety over innovation to a degree where it kills in my view kills safety in the long term so the counterfactual thing they just just actually exploring this world of how do you interact with dynamic objects and so on how do you how do you think about safety you can do reinforcement learning without ever exploring and I said that like so you can think about you're in like a reinforcement learning it's usually called like a temperature parameter and your temperature parameter is how often you deviate from the Arg max I could always set that to zero and still learn and I feel that you'd always want that set to zero on your actual system got you but the problem is you first don't know very much and so you're going to make mistakes so the learning the exploration happens to ready yeah but okay so the consequences of a mistake yeah open pilot and autopilot are making mistakes left and right yeah we have we have we have 700 daily active users a thousand weekly active users open pilot makes tens of thousands of mistakes a week these mistakes have zero consequences these mistakes are oh it I wanted to take this exit and it went straight so I'm just gonna carefully touch the wheel humans the humans catch them and the human disengagement is labeling that reinforcement learning in a completely consequence-free way so driver monitoring is the way you ensure they keep yes they keep paying attention how is your messaging say I gave you a billion dollars you would be scaling and now oh my fact it's guy couldn't scale with any amount of money I'd raise money if I could if I had way to scale yeah you're not focusing I don't know I don't know how to do Oh like I guess I could sell it to more people but I want to make the system better better I don't know I mean but what's the messaging here I got a chance to talk to you on and and he he basically said that the human factor doesn't matter you know the human doesn't matter because the system will perform there would be sort of a sorry to use the term but like a singular like a point where it gets just much better and so the human it won't won't really matter but it seems like that human caching the system when it gets into trouble is like the thing which will make something like reinforcement learning work so how do you how do you think messaging for Tesla for you should chant for the industry in general should change I think my messaging is pretty clear at least like our messaging wasn't that clear in the beginning and I do kind of fault myself for that we are proud right now to be a level 2 system we are proud to be level 2 if we talk about level 4 it's not what the current hardware it's not gonna be just a magical OTA upgrade it's gonna be new hardware it's gonna be very carefully thought-out right now we are proud to be level 2 and we have a rigorous safety model I mean not like like okay rigorous who knows what that means but we at least have a safety model and we make it explicit is in safety MD and open pilot and it says seriously though safety dot MD Android so well this is this is the safety model and I like to have conversations like if like you know sometimes people will come to you and they're like your systems not safe okay have you read my safety Doc's would you like to have an intelligent conversation about this and the answer is always no they just like scream about it runs Python okay what so you're saying that that because pythons not real-time Python not being real-time never causes disengagement disengagement SAR caused by you know the model is QM but safety dad MD says the following first and foremost the driver must be paying attention at all times I don't can I do I still consider the software to be alpha software until we can actually enforce that statement but I feel it's very well communicated to our users two more things one is the user must be able to easily take control of the vehicle at all times mm-hmm so if you step on the gas or brake with open pilot it gives full manual control back to the user or press the cancel button step 2 the car will never react so quickly we define so quickly to be about one second that you can't react in time and we do this by enforcing torque limits braking limits and acceleration limits so we have um like our torque limits way lower than Tesla's this is another potential if I could tweak autopilot I would lower their torque limit or would a driver monitoring um because autopilot can jerk the wheel hard yeah open pilot can it's we we limit um and all this code is open source readable and I believe now it's all misery C compliant misra is like the automotive coding standard um at first I you know I've come to respect I've been reading like the standards lately and I've come to respect them they're actually written by very smart people yeah they're brilliant people actually they have a lot of experience there's sometimes a little too cautious but in this case it pays off miss was written by like computer scientists and you tell them as a language they use you can tell by the language they use they talk about like whether certain conditions in misra are decidable or undecidable you mean like the halting problem and yes well all right you've earned my respect I will tell you carefully what you have to say and we want to make our code compliant with that all right so you're proud level two and reform so you were the founder and I think CEO of comm AI then you were the head of research what the heck are you know what's your connection to come AI the president but I'm one of those like unelect unelected presidents of like like a small dictatorship country not one of those like elected presidents oh so you're like Putin when he was like yeah I got sure so there's uh what's the governance structure what's the what's the future of commie I finance I mean as a business do you want you just focused on getting things right now making some small amount of money and mean to and then one that works it works in each scale our burn rate is about 200k a month and our revenue is about 100k a month so we need to 4x our revenue but uh we haven't like tried very hard at that yet and the revenue is basically selling stuff online yeah we sell stuff shopped a comment at AI is there other well okay so you you'll have to figure out that's our that's our only see but to me that's like respectable revenues yeah we make it by selling products to consumers we're honest and transparent about what they are most actually level for companies right because you could easily start blowing up like smoke like over selling the hype and feeding into getting some fundraisers oh you're the guy you're genius because you hacked the iPhone oh I hate that I hate that yeah I can trade my social capital for more money yeah I did it once I almost regret it doing the first of it well on a small tangent what's your you seem to not like Fame and yet you're also drawn to fame what were you on we're on you where are you on that currently have you had some introspection some soul-searching yeah I actually I've come to a pretty stable position on that like after the first time I realized that I don't want attention from the masses I want attention from people who I respect who you respect I can give a list of people so are these like Elon must have characters yeah well actually you know what I'll make it more broad than that I won't make it about a person I respect skill I respect people who have skills right and I would like to like be I'm not gonna say famous but be like known among more people who have like real skills who in cars doers do you think have skill not do you respect Oh Kyle vote has skill a lot of people away mo have skill and I respect them I I respect them as engineers like I can think I mean I think about all the times in my life where I've been like dead set on approaches and they turn out to be wrong so I mean this might I might be wrong I accept that I accept that there's a decent chance that I'm I'm wrong and actually I mean having talked to Chris Urmson sterling anderson i those those guys I mean I deeply respect Chris I just admire the guy he's legit can you drive a car through the desert when everybody thinks it's impossible that is that's legit and then I also really respect the people who are like writing the infrastructure of the world like the linus torvalds and the chris lab they're doing the real work I know they're doing the real work this every dog that Chris Ladin you realize especially when they're humble it's like you realize oh you guys were just using your oh yeah all the hard work they did him that's incredible what do you think mr. Anthony lowendahl ski what do you he's a he's another mad genius sharp guy oh yeah what do you think he might long-term become a competitor Oh tu cama well so I think that he has the other right approach I think that right now there's two right approaches one is what we're doing and one is what he's doing can you describe I think it's called pronto a certain you thing did do you know what what the approaches actually don't know embark is also doing the same sort of thing the idea is almost that you want to so if you're I can't partner with Honda and Toyota Honda and Toyota are uh like four hundred thousand person companies it's not even a company at that point like I don't think of it like I don't personify it I think of it like an object but a trucker drives for a fleet maybe that has like some truckers are independent some truckers Drive for fleets with a hundred trucks there are tons of independent trucking companies out there start a trucking company and drive your costs down or figure out how to drive down the cost of trucking another company that I really respect is uh not oh I should I respect their business model no auto sells a driver monitoring camera and they sell it to fleet owners if I that's right if I owned a fleet of cars and I could pay you know 40 bucks a month to monitor my employees this is gonna like reduces accidents 18% yeah it's it's so like that in the space that is like the business model that I like most respect is there creating value today yeah which is uh that's a huge one is how do we create value today with some of this then the link keeping things huge and it sounds like you're creeping in or full steam ahead on the driver monitoring - yeah which I think actually were the short-term value if you can get right I still I'm not a huge fan of the statement that everything is to have driver monitoring but will I agree with that completely but I'm that statement usually misses the point that to get the experience of it right is not trivial oh no not at all in fact like so right now we have I think the time out depends on speed of the car but we want to depend on like the scenes day if you're on like an empty Highway it's very different if you don't pay attention then if light you're like coming up to a traffic light and long-term it should probably learn from from the driver because that's to do I watched a lot of video we've built a smartphone detector just to analyze how people are using smartphones and people are using it very differently and there's a it's a texting styles there's videos yeah like I got billions of miles of people driving cars in this moment I spent a large fraction of my time just watching videos because it's never fails to to learn like it never I've never failed from a video watching session to learn something I didn't know before fact I usually like when I eat lunch I'll sit especially when the weather is good and just watch pedestrians with an eye to understand like from a computer vision I just to see can this model can you predict what are the decisions made and there's so many things that we don't understand this is what I mean about the state vector yeah it's I'm trying to always think like Gamma understanding in my human brain how do we convert that into how hard is the learning problem here I guess is the fundamental question so something that from a hacking perspective this is always comes up especially with folks well first the most popular question is the trolley problem right so that's not a sort of a serious problem there are some ethical questions I think that arise maybe will you want to met you or do you think there's any ethical serious ethical questions that we have a solution to the trolley problem Akane aye well so there is actually an alert in our code ethical dilemma detected it's not triggered yeah we don't we don't how you have to detect the ethical dilemmas but we're a level two system so we're going to disengage and leave that decision to the human you're such a troll hey no but the trolley problem deserves to be trolled yeah that's a beautiful answer actually I know I gave it to someone who was like sometimes people ask like you asked about the trolley problems like you can have a kind of discussion about it like boo you get someone who's like really like earnest about it because it's the kind of thing where if you ask a bunch of people in an office whether we should use a sequal stack or no sequel stack if they're not that technical they have no opinion but if you ask them what color they want to paint the office everyone has an opinion on that and that's why the trolley problem is that's it I mean it's a beautiful answer yeah we're able to detect the problem and were able to pass it on to the human yeah I've never never heard anyone say it nice escape route okay but proud level - I'm proud level - I love it so the other thing that people cope you know have some concern about with AI in general is hacking so how hard is it do you think to hack a nataas vehicle either through physical access or through the more sort of popular now these adversarial examples on the sensors be adversarial examples one you want to see some adversarial examples that affect humans hmm right oh well there used to be a stop sign here but I put a black bag over the stop sign and then people ran it all right adversarial yeah right like like like there's tons of human adversarial examples - um the question in general about like security if you saw something something just came out today I'm like there are always such high P headlines about like how navigate on autopilot was fooled by a GPS spoof to take an exit right at least that's all they could do was take an exit if your car is relying on GPS in order to have a safe driving policy they're doing something if you're relying and this is why v2v is such a terrible idea v2v now relies on both parties getting communication right this is not even so I think of safety security is like a special case of safety right safety is like we put a little you know piece of caution tape around the hole so that people won't walk into it by accident security is I put a 10 foot fence around the hole so you actually physically cannot climb into it with barbed wire on the top and stuff right so like if you're designing systems that are like unreliable they're definitely not secure your car should always do something safe using its local sensors and then the local sensor should be hardwired and then could somebody hack into your can boss and turn your steering wheel on your brakes yes but they could do it before common AI too so let's think out of the box and some things so do you think teleoperation has a role in any of this so remotely stepping in and controlling the cars no I think that if safety if the safety operation by design requires a constant link to the cars I think it doesn't work so that's the same argument using for v2i VTV well there's a lot of non safety critical stuff you can do with v2 I like v2 I liked v2 I weigh more than V B because Vita I is is already like I already have internet in the car right there's a lot of great stuff you can do with v2 I um like for example you can well where I already have v2 Waze is V die right ways can route me around traffic jams that's a great example of v2 I mm-hmm and then okay the car automatically talks to that same service like improving the experience but it's not a fundamental fallback for safety know if any of your if any of your if any of your things that require wireless communication are more than qm like have a nozzle rating you should you previously said that life is work and then you don't do anything to relax so how do you think about hard work well what is it what do you think it takes to as great things you know there's a lot of people saying that there needs to be some balance you know you need to in order to accomplish great things you need to take some time off each of reflects and so on now and then some people are just insanely working burning the candle at both ends how do you think about that I think I was trolling in the Siraj interview when I said that off camera right before I spoke a little bit we'd like get out spot this is a joke right like I do nothing it relaxed look where I am I'm at a party right yeah that's true so no of course I I don't um what I say that life is work though I mean that like I think that what gives my life meaning is work I don't mean that every minute of the day you should be working I actually think this is not the best way to maximize results I think that if you're working 12 hours a day you should be working smarter and not harder well so it gives work gives you meaning for some people other source of meaning is personal relationships yeah like family and so on you've also in that interview of Sirach or does the the trolling mentioned that one of the things you look forward to in the future is AI girlfriends yes so at the topic that I'm all very much fascinated by not necessarily girlfriends but just forming a deep connection with AI what kind of system do you imagine when you say AI girlfriend whether you were trolling or not know that one I'm very serious about and I'm serious about that on both a shallow level and a deep level I think that VR brothels are coming soon and are gonna be really cool it's not cheating if it's a robot I see the slogan already but there's I don't know if you've watched it just watched the black mirror episode i watch the one year yeah yeah oh the the Ashley - one way da no where there's two friends were having sex with each other and mo in the VR game your game it's just two guys but yeah one of them was was a female and yeah there's another mind-blowing concept that in VR you don't have to be the form you can be to animals having sex weird I mean I'll see you I said the software Maps the nerve endings right yeah yeah they they sweep a lot of the fascinating really difficult technical challenges under the rock like assuming it's possible to do the mapping of the nerve endings then I wish yeah I saw that the way they did it with a little like stim unit on the head that'd be amazing so wanna know on a shallow level like you could set up like almost a brothel with like real dolls and oculus quests right some good software I think it vehicle novelty experience you know on a deeper like emotional level I mean yeah I would really like to fall in love with with with the machine do you see yourself having a long-term relationship of the kind monogamous relationship that we have now with a robot with a a AI system even not even just the robot so I think about maybe my ideal future when I was fifteen I read eliezer yudkowsky early writings mmm-hmm on the singularity and like that AI is going to surpass human intelligence massively he made some Moore's law based predictions that I mostly agree with and then I really struggled for the next couple years of my life like why should I even bother to learn anything it's all gonna be meaningless when the machines show up right maybe maybe when I was that young I was still a little bit more pure and really like clung to that and I'm like wow the machines ain't here yet you know and I seem to be pretty good at this stuff let's uh let's try my best you know like what's the worst that happens but the best possible future I see is me sort of merging with the Machine and the way that I personify this is in a long-term monogamous relationship with a machine oh you don't think there's room for another human in your life if you really truly merge with another machine I mean I see merging I see like the best interface to my brain is like the same relationship and to merge with an AI right does that merging feel like I see yeah I've seen couples who've been together for a long time and like I almost think of them as one person like couples who spend all their time together and that's that's how you're actually putting what does that merging actually looks like it's not just a nice channel like a lot of people imagine it's just an efficient link search link to Wikipedia or something I don't believe in that but it's more you're saying that there's the same kind of the same kind of relationship you have one other human that's a deep relationship is that's what merging looks like that's that's pretty uh I don't believe that link is possible um I think that that link so you're like oh me to download Wikipedia right to my brain yeah my reading speed is not limited by my eyes my reading speed is limited by my inner processing locally and to like bootstrap that sounds kind of unclear how to do it and horrify but if I am with somebody and I'll use a somebody who is making a super sophisticated model of me and then running simulations on that model I'm not gonna get into the question whether the simulations are conscious or not I don't really want to know what it's doing um but using those simulations to play out hypothetical futures for me deciding what things to say to me to guide me along a path and that's how I envision it so on that path to AI of superhuman level intelligence you've mentioned that you believe in the singularity that singularity is coming yeah again could be trolling could be not could be part I'm all trolling his truth in it I don't know what that means anymore what is the singularity yeah so that's that's really the question how many years do you think before the singularity what form do you think it will take does that mean fundamental shifts and capabilities of AI does it mean some other kind of ideas um maybe this is just my roots but so I can buy a human beings worth of compute for like a million bucks that I it's about one TPU pod v3 I want like I think they claim a hundred peda flops that's being generous I think humans are actually more like twenty so that's like five humans that's pretty good Google needs to sell their teepees um but I could buy I could buy I could buy GPUs I could buy a stack of like by 1080 tea eyes build data center full of them four million box I can get a human worth of compute but when you look at the total number of flops in the world when you look at human flops which goes up very very slowly with the population and machine flops which goes up exponentially but it's still nowhere near I think that's the key thing to talk about when the singularity happened when most flops in the world are silicon and not biological that's kind of the crossing point like they are now the dominant species on the planet and just looking at how technology is progressing when do you think that could possibly happen you think go to happen in your lifetime oh yeah definitely my lifetime I've done the math I like 2038 because it's the UNIX timestamp rollover yeah beautifully put so you've you said that the meaning of life has to win if you look five years into the future what does winning look like so [Music] hi there's a lot of I can go into like technical depth to what I mean by that to win um it may not mean I was criticized for that in the comments like doesn't this guy want to like save the penguins in Antarctica or like you know listen to what I'm saying I'm not talking about like I have a yacht or something I am an agent I am put into this world and I don't really know what my purpose is but if you're a reinforcement if you're if you're an intelligent agent and you're put into a world what is the ideal thing to do well the ideal thing mathematically you go back to like Schmitt Hoover theories about this is to build a compressive model of the world to build a maximally compressive to explore the world such that your exploration function maximizes the derivative of compression of the past mid Hoover has a paper about this and like I took that kind of as like a personal goal function so what I mean to win I mean like maybe maybe this is religious but like I think that in the future I might be given a real purpose or I may decide this purpose myself and then at that point now I know what the game is and I know how to win I think right now I'm still just trying to figure out what the game is but once I know so you have you have imperfect information you have a lot of uncertainty about the reward function and you're discovering it exactly the purpose is that's that's the better way to put it the purpose is to maximize it while you have it a lot of uncertainty around it and you're both reducing the uncertainty and maximizing at the same time yeah and so that's at the technical level what is the if you believe in the universal prior yeah what is the universal reward function that's the better way to put it so that when it's interesting I think I speak for everyone in saying that I wonder what that reward function is for you and I look forward to seeing that in five years in ten years I think a lot of people who do myself right and cheering you on man so I'm I'm a happy you exist and I wish you the best of luck thanks for talking today man thank you this is a lot of fun you English (auto-generated)