Is and ought

I have been watching some of the videos from the Edge seminar THE NEW SCIENCE OF MORALITY. There will eventually be about 10 hours of talks and discussions posted on the Edge site. From the few presentations I have seen so far this looks to have been a fascinating seminar.

Partly because the science is relatively new – but also because there has been a lot of progress made. However, there are of course areas which promote intense discussion. I get the impression, for example, that several of the participants wish to challenge to dogma that one can’t determine an ought from an is. It’s going to be interesting to see that debate played out.

WEIRD culture and reasoning

Jonathon Haidt

Jonathon Haidt was the first speaker and made some interesting points about the relevance of a science centred largely around specialists from advanced western countries. He is using the acronym WIERD for the orientation around cultures in the Western, Educated, Industrialized, Rich and Democratic (WEIRD) societies. This analysis comes from a recent paper The weirdest people in the world ? by J Henrich, SJ Heine and A Norenzayan. Those authors say “we need to be less cavalier in addressing questions of human nature, on the basis of data drawn from this particularly thin and rather unusual slice of humanity.”

Haidt also discusses some fallacies about human reasoning. “The puzzle is, why are humans so amazingly bad at reasoning in some contexts, and so amazingly good in others?” Again he refers to a recent paper by Mercier and Sperber – Why do humans reason ? Arguments for an argumentative theory. This is an interesting paper discussing human problems like confirmation bias, the human problem of search for evidence to support an preconceived conclusion.

Obviously both these problems are very relevant to a seminar like this. Go to the Edge site for a video of Jonothan’s presentation or download and audio file (MP3 Audio Download β€” Jonathan Haidt Talk).

Sam Harris and a role for science

As Sam Harris was one of the participants the seminar will surely have also discussed his ideas on the role of science in determining right and wrong. He presented these ideas in two recent lectures and they resulted in a lot of discussion, and controversy, on a number of scientific blogs (see Can science answer moral questions? and


Similar articles

Enhanced by Zemanta


38 responses to “Is and ought

  1. Pingback: Top Seven « Tangled Shoelaces

  2. I have just started watching the video from is. I really enjoyed the talk froM Jonathan Haidt, and am looking forward to the rest.

    A little side story here is that I have recently started reading some philosophy texts. This is because I have bought and IPad and found it comfortable enough to read on to actually read books, normally I would not attempt this on the computer. So, I have downloaded heaps of books for free from project Gutenberg. Hulm, Kant etc..

    I have so far read one of Hulms books, not on morals, but humzn reasoning, and enjoyed it much more than I thought I was going to, in fact I was pleasantly surprised about how muchmhe understood and factored in an understandingmof human failings, this being a subject that Is near to my heart when it comes to philosophy. I also found a lot of his ideas echoed at least a little evoloutinary thinking, so was quite pleased to hear Haidt’s reference to Hulm in his talk. I am now reading one of Kant’s books and am finding this a bit more heavy going, but am attempting to understand where he was coming from, even if I disagree.


  3. Sounds like you are finding the iPad useful, Nick. I suspect you typed your comment on it because it has similar mistakes to the ones I make with my iPod Touch. Presumably this is related to the auto-correction and on-screen keyboard rather than just the size of my fingers.

    It’s good that these classics are so readily available once we have a good book reader. I have several on my iPod Touch but will really only read from that if there are no alternatives.

    There seems to be a feeling that the iPad will really push the price of eBook Readers down. A local book blog even suggested that in a few years the readers will be about the same price as a print book!

    In NZ Whitcoulls launched the Kobo reader a month or so back. And it looks like we will soon get the new Kindle via Vodophone.

    I guess the iPad is not as portable as the readers or as suitable for reading in sunlight. But I will be intrigued to hear if the bright screen is at all a disadvantage for comfortable reading in normal situations.


  4. Yep, those type failures are indeed iPad related. Perhaps when I have a bit more practice with the virtual keyboard I will be better. Actually, the main problem is not the auto correction, rather that I have not yet found a backspace key (rather than delete) on the keyboard, and I find the touch screen cursor locate a bit too time consuming to use. The combination of these factors and my innate laziness mean that I tend to leave some errors rather than correct them, as long as the comprehension is not too badly affected.

    I have been using the Stanza app for reading, and have swapped this to night mode, so that is white on black, and find this surprisingly good for reading, I have now read around three complete. Books like this. I think I have done some reading outside, and its a bit harder but ok, but I have been on holiday for the last couple of weeks, and it is a definite disadvantage when it comes to reading down at the beach. I would not use the iPad for this, so this can be a bit frustating as I would like to read my current book.

    Ps. I really enjoyed the Joshua Green talk also.


  5. Wow, I just watched the Paul Bloom talk. I seriously recommend this one to everyone.

    I have often, somewhat ashamedly, wondered about how much of the shift in public thinking in terms of things like discrimination against gay people can be put down to shortland street plot lines. One real possibility here is that this is the mechanism whereby moral rationality becomes codified into moral intuition.

    On the dark side of this thought, compare modern american shows like 24 etc… to past shows like M.A.S.H. This does not give me much optimism.


  6. I am using the last day of my holiday to watch all these videos, it’s not that I have a lot of time on my hands …:-)


  7. Just watched the last one, Joshua Knobe. I really recommend this one to people like Dale that pop up here now and again. This talk is broadly about the concept of experimental philosophy and I think hints at some really deep issues that explain the recurring debates and fundamental divides between different people.

    It seems that the discussion sections of these talks have not been posted yet, and I am really keen to see what this was like.


  8. Nick, the discussion videos of the the first two presentations (Haidt and Greene) are up. But it looks like Edge may be putting the rest into separate posts over the next month. They say:

    “Over the next month we will serialize the conference by rolling out one or two of 45-minute sessions as an Edge Edition. This will include HD video of the 25-minute talk (with complete text), the 20-minute discussion, and a doublowdable audio MP3 of the talk. We will end the series with the last two ninety-minute discussions on “The New Science of Morality”. “

    Thanks for reminding me of Bloom’s point about the role of media. I agree with you about 24. I think I have noticed justification of torture creeping into other TV dramas over the last few years as well.


  9. How embarassing, I have of course consistently misspelt David Hume’s name in my previous comments. I blame it on the stress of becoming multi lingual πŸ˜‰

    I have yesterday watched the discussion after the Sam Harris talk and am interested in hearing other peoples impressions on his talk.

    I was a little less uncomfortable with this talk as opposed to previous from Harris, perhaps because he included less examples of what he considered to be the obvious moral judgements that science can already make. What is interesting however, is that I find it difficult to nail down what it is about Harris that makes me uncomfortable. Afte the discussiin I got the feeling that this might also be the case for some of the participants in this workshop also.

    Anyway, one point that I picked up on was his dismissal of the research into the evoloutinary basis of humans as ‘interesting’. I would of thought that is would be absolutely central to finding out what constitutes human well being. I mean, we are the product of our evolution, and as far as there is an emotional component of well being, emotions (any feelings really) that we feel must be a product of the developmental history of our species.

    This point actually raises a number of fundamental problems with the idea of ‘obviously bad’ moral strategies. I.e. I can easily hypothesise specieies of concious beings that that have a quite different or incompatible set of evolutionary emotional adaptations. What would we make of an concious entity with a background not so rich in social cooperation as us. Perhaps instead, a highle competitive tooth and nail history. How can we find our moral certainty if for such a hypothetical creature fighting and killing were fundamental components of its well being.


  10. I am afraid you are ahead of me, Nick. I have yet to watch the discussions.

    I think I can appreciate your unease with Harris. However, I think he is of value on raising the questions he does – even if he is wrong in some ways or is being a bit arrogant.

    I am looking forward to his book and while I realize one can’t place too much value on cover recommendations he does have some good names. And I think some of these are indicating support for his challenge to the is-ought dogmatic interpretation.

    He is clearly challenging ideas of moral relativism (not that they are cropping up in this seminar) and I do like the idea that we can have an objective basis for our moral judgements.

    Re your idea of a different intelligent species with different moral views. While one could easily hypothesize this I wonder to what extent such a species could really exist. Given the similarities in the emotions and intuitions of other close primates I wonder to what extent a species could be very different morally if it evolved as an intelligent, social and empathetic species. And it seems to me that an intelligent species would also inevitably be social and empathetic.

    After all, fighting and killing are and were characteristics of our species. It seems to me that an alternative social species would have to find ways of dealing with this once it’s social units got beyond a certain size.

    Even within the current human species different social evolutions has produced variation in moral values. And these have also changed with time.

    But, I think that the nature of our existence as an intelligent, social empathetic species can provide a factual basis for building a common morality. While there are variations between different societies and cultures these facts mean that we can in the end come to agreement on some common moral judgements.

    But I am intrigued by Sam’s idea of moral landscape where different groups may develop different acceptable and successful moralities.

    I must catch up with the discussion videos. The whole seminar seems to be very valuable and I hope more people will actually watch the videos or listen to the mp3 files. It provides a lot more meat for discussion than silly arguments that our morals come from a god.


  11. I agree that this is a valuable discussion to raise.

    After a bit more reflection, I think the bit that I am most uneasy about is where Harris says that just accepting that there is an absolute right or wrong is the most important thing. I personally don’t see that as important at all.

    He speaks very eloquently and persuasively about the idea of a moral landscape and how we can and should attempt to use rational investigation to underpin our morality and actions, and I fully agree.

    I, however, would completely avoid the whole absolute right/wrong is/ought issue, by couching the discussion explicitly in terms of the actions and outcomes. In other words, justifying a particular course of action with a rational argument based on empirical facts and probable outcomes. This can of course include argumentation as to why you think that the particular expected outcome is better than other outcomes that would result from other actions or inaction. I don’t see that ‘defining’ this as the right thing in an absolute objective sense adds anything at all to the original process. In fact, this smacks of ‘moral certainty’ which is not what I would expect a careful and empirical reasoning process would lead. What impresses me most is when people are explicit about the uncertainties that their research has shown them.


  12. In my earlier example, I was trying to posit a non social cognitive or intelligent entity.

    I personally would like to think that social behavior is a strongly beneficial evolutionary adaption and perhaps even necessary to reach higher levels of cognitive development, but don’t think I could rule out high levels of richness or complexity of cognition and emotion being reached under a completely different basis. What I actually had in mind when I thought of this was a particular type of shark which gestates multiple fetuses at the same time, but where only single births occur, as the young sharks fight and eat each other in utero until there is only one left. (Warning, I suspect that I saw this on some bad reality TV show, so this might be utter bollocks).


  13. Sure, one can provided reasoned augments for moral judgements avoiding “right”, “wrong” etc. But humans don’t normally do that. Commonly we talk about things being “right” or “wrong.” And if we supply reasoning it is usually after the fact.

    Perhaps because we have evolved to react quickly and intuitively, rather than after careful consideration (by which time the sabre tooth tiger has eaten us) we also have an intuitive concept of “right” and “wrong.” Or an intuitive need to use such labels in justification. Of course this provides opportunities for religions and ideologies to co-opt our intuitions and use them to justify their myths.

    I think we do, and have, made progress by moral reasoning and logic. Also by better communication and contact. But, again, I don’t think we then continue to use such reasoning. it’s like learning to ride a bike. Our reasoned arguments produce moral decisions which then become part of our subconsciousness. Again we are back into the position of saying it is “wrong” to treat women as second class citizens, be homophobic or keep slaves. Whereas in the past we thought these things were “right.”

    We might actually have a hard time providing a reasoned argument for our new moral judgments, or get into rationalisation more than objective thinking.

    Anyway, this is the sort of message I got from several of the Seminar speakers.

    I agree that it is a trap to talk about absolute morality – after all religions like to do that for their own reasons. I prefer to think there is an objective basis for moral decisions/judgements. This enables us to reach similar, or the same, conclusions from considering evidence. I agree that there must be room for uncertainties and variability.

    “Absolute morality” seems to lend itself to imposition from above. In my view this makes it hard for individuals to build their own moral logic and intuitions. They can end up appealing to parents, leaders, priests, etc., instead.


  14. Nick:

    I was a little less uncomfortable with this talk as opposed to previous from Harris, perhaps because he included less examples of what he considered to be the obvious moral judgements that science can already make. What is interesting however, is that I find it difficult to nail down what it is about Harris that makes me uncomfortable.

    I suspect it’s something like this…

    Hume, when talking about the is-ought problem, didn’t so much state that is->ought is a fallacy, but that when someone claims that an is justifies an ought, there is a hidden proscriptive premise. The person making the argument may be unaware that they are doing it, or may be doing it deliberately to hide a political axe they are grinding or what not.

    The danger is in any case, that the hidden value goes unanalysed. It’s a failure of critical thinking.

    The problem with Sam’s examples of moral facts, is because he doesn’t acknowledge, or even seem to recognise the prescriptive premise that underpins his examples of science determining moral values.

    For example, when Sam talks about us not feeling anything about rocks, and less about ants than primates, he states that this stems from our factual recognition of rocks not being able to suffer and ants being less capable of suffering than primates. This is a good example of Sam getting it wrong – this continuum of suffering would be incapable of moral implications if it first did not presuppose that suffering was an immoral outcome – a value statement.

    This becomes apparent when you ask the question “why is suffering bad?” Sam has only been talking about the quantification of suffering, not why suffering is bad per se. This is a (barely) hidden premise in Sam’s argument; it’s not the case that Sam has gone from is-to-ought and failed, it’s that he’s gone form ought-to-ought and not realised it.

    I’ve suspected that Sam’s problem with Hume, Moore (and Popper) has been one of not really quite getting them. His degree in philosophy not withstanding.

    I covered Hume, Popper and Moore in the ethics/critical thinking component of my science degree, and it’s surprising what can get past the goalposts with a good mark at an undergrad level. I can’t imagine Sam spending much more time on the topic as I did, given all the other areas of philosophy he’d have to have covered in his degree.

    It is of course possible to be a bit loose in a certain area and be a good philosopher, but if Sam wants to say something about the is-ought problem, he really needs to tighten it up a bit.

    I don’t have a problem with him proposing that science is crucially important to all the vexing moral issues. If you presuppose that suffering is bad and hence prescribe against it, you need to quantify suffering in order to make specific moral claims after all.

    Indeed, I suspect I wouldn’t discard any of Sam’s scientific content as being irrelevant to ethical inquiry, at least not on philosophical grounds. (Standard scientific criticism would apply).

    But there is a difference between science being essential to moral policy making by way of informing the decision making, and science prescribing moral outcomes. It’s the philosophical values that prescribe and the science that describes.

    If there’s one thing at the bottom of all this confusion that I think irks me the most though, is that I’ve been arguing with creationists, apologists et. al. over this for years; having to point out to them that science can have a role in ethics without posing an is-ought problem. Then in comes Sam, trying to demolish Hume out of the same misguided belief possessed by religious apologists – hence ceding a point to them unnecessarily.

    The myth about is-ought is that is-ought prevents science and ethics from ever touching. It doesn’t. That’s the myth I wish Sam would take on, rather than try to take down what is a much misunderstood, but important contribution to Enlightenment critical thinking.

    Anyway, that’s my take on Harris and Hume. These are of course somewhat tentative – I haven’t read Sam’s new book obviously, but his public talks on the matter aren’t encouraging re: is-ought.


  15. Gah. Typos and grammatical errors. Long comments are so hard to edit in a small box… 😦


  16. Bruce, on the surface I agree that there is a value assumption that suffering is wrong. But I can’t see why we don’t just accept that as a given and get on to solving real problems. I think that when people get into arguments demanding that we “prove” something like this logically or philosophically we are warranted in our reaction to walk away and leave them to it.

    To me we don’t have to use logical or philosophical proofs for something that obvious. And it is obvious because it is within all of us. Our brains are wired that way. If they weren’t our species probably would not have survived.

    Similarly I think it is perfectly natural that we are wired to be empathetic. And our existence as a social species means that we are also hardwired with some basic intuitions. These form a basis for our ability to react instinctively on many moral issues.

    I agree that Sam hasn’t been very clear on this current science/morality issue – although the discussion at this seminar has helped. And I think it is always worth having voices like his in such discussions. I am hoping his writing will be clearer.

    It’s interesting though how he is provoking a number of fearful reactions. One I am conscious of in myself is the (probably unwarranted) extrapolation to a situation where society will be told what is right and wrong, and what is correct, down to details by scientists. The discussion on what percentage of one’s income should be donated to charity is an example. (Most of us font like “shoulds”).

    Of course we don’t want such a brave new world society. We only have to think back to eugenics. We don’t want science controlling society like that, nor should we want philosophers, ethicists, clerics, etc. with that sort of control.

    While I say this fear may be unwarranted I am also aware that authorities will take a mile when they are offered an inch. The way the UK and USA have used the terrorist threat to reduce personal freedoms and rights does make me fearful. I don’t want any “experts” reorganizing society because they claim it is in my best interests.

    I suspect it would be better to understand the term flourishing in a more general and less quantitative way. I think it is important to fight against moral relativism, to recognize there are some things humans do to each other which are clearly wrong because they cause suffering. But we don’t want to have happiness imposed on us. Rather we want to remove suffering, provide basic human rights and freedoms, provide opportunities for all, etc. And then just let people get on with it.

    After all, some people aren’t happy unless they unhappy. Forcing happiness on them won’t work.


  17. Bruce, on the surface I agree that there is a value assumption that suffering is wrong. But I can’t see why we don’t just accept that as a given and get on to solving real problems. I think that when people get into arguments demanding that we β€œprove” something like this logically or philosophically we are warranted in our reaction to walk away and leave them to it.

    Well, whether or not suffering is immoral being somewhat beside the point I’m making, I’ll take this opportunity to disagree with you to some extent. πŸ˜€

    To me we don’t have to use logical or philosophical proofs for something that obvious. And it is obvious because it is within all of us. Our brains are wired that way. If they weren’t our species probably would not have survived.

    Not entirely dissimilar to how the capacity for rape is wired into us. Or any number of urges we have that have been discarded as base, and perhaps a few more that we currently take for granted.

    Nutting these things out requires that we explicitly analyse our values, even if it is a bit obvious at times, if only because it’s not obvious at other times (which brings me back to what I’m on about is-ought for).

    Take for example the values of economic rationalism that have managed to sell themselves. We are told X is a more economical proposition, and that has for a large part of political discourse in the past thirty years, been that. If democracies had been more vigilant in weeding out the unstated values of dog-eat-dog, we could have had a more explicit discussion of dog-eat-dog versus improving humanity’s lot.

    This isn’t to say that the facts of good economics aren’t relevant, obviously they are. It’s just that there hasn’t been enough discussion of what it’s all been instrumental to.

    (I’d argue that this is-ought problem is also evident in Australia’s immigration policy – nobody questions why boat people are bad, but people who let their visas expire don’t matter).

    Am finding myself agreeing with your words on unwarranted fears, liberty and relativism. Although I’ll add one caveat; a worry I have had about Harris since The End of Faith, is that I worry he’s going to sell the idea that relativism is a necessary product of Kuhn, Popper, Moore and Hume, which in my experience has always been a product of these philosophers being misunderstood*. Either by their detractors, or by relativists seeking to use them as justification.

    After all, some people aren’t happy unless they unhappy. Forcing happiness on them won’t work.

    Or that some things bring happiness for some people, whereas they don’t for others. Some people like it up the butt**.

    * For example, “edifice collapse”, the phenomena of a body of knowledge collapsing when a fundamental scientific assumption is falsified ala Popper, is based on a misunderstanding of Popper. Rejection is an ideal consequence of falsification, but is not necessary, esp. if their is no hypothesis/theory that can substitute. A falsified hypothesis still being worth more that no hypothesis, or an unfalsifiable hypothesis. “Edifice collapse” is just a strawman to make scientists look like they hold onto orthodox hypothesis as a matter of cultural norm, when it’s there in The Logic of Scientific Discovery, the reasons why a hypothesis may not be rejected when it is falsified, and they’ve got nothing to do with cultural norms.

    ** I’ve got an thought experiment brewing along these lines as to why the categorical imperative is not the same as the golden rule. But that’s another story. πŸ˜›


  18. Bruce, we may be talking past each other. I thought you made the point that judging suffering to be wrong is an assumed value statement which Sam had not proved. My point was that proving is not necessary. We just define good and bad this way. I have noticed that the theologically inclined argue for a god given good and bad and hence atheists have no justification for making moral judgements. I think that is a dishonest argument – and typically attempts to trap people into jelly wrestling. The sensible people have no problem knowing that suffering is bad and just get on with their lives. We don’t need theologians to tell us what their god thinks us right and wrong.

    I don’t agree that rape is hard wired. We have natural instincts, urges, intuitions, and these can of course lead us to do things that cause ourselves or others to suffer. By definition those actions are wrong.

    The underlying intuitions, instincts, urges, are hardwired. But the actions themselves are not hardwired. And we can usually come to the conclusion these actions are wrong by ourselves.

    We can make instant moral judgements about right and wrong and choose whether to go ahead with an action. We can make judgements because we are empathetic. That is hardwired. And yes we can also apply reason to make such moral decisions – although that takes time. Something we don’t always have.

    I won’t at this stage comment on your references to Kuhn and Popper. I do feel both are often used incorrectly to make misleading statements about the science process. Currently I am reading Alan Chalmers on them and can see that a straight philosophical application of their ideas to science is too naive. It certainly doesn’t correspond to my understanding of scientific methodology from experience in research.

    I think Chalmer’s criticisms of the two are quite appropriate. He also has the advantage of taking all the different philosophical contributions, pointing out their problems but still taking on the positive aspects.


  19. Bruce, we may be talking past each other. I thought you made the point that judging suffering to be wrong is an assumed value statement which Sam had not proved.

    No, my point was that “suffering is wrong” wasn’t a properly acknowledged assumption. Sam has been saying that it’s possible to go from an is-to-an-ought, which is invalid, plus all of his examples of supposed is-to-ought, have in line with Hume’s observation been the result of failing to acknowledge an assumed value.

    Sam’s been going from ought-to-ought, ala:

    P1: We should be concerned about wellbeing where there is suffering. (Value)
    P2: Rocks can’t suffer. (Fact)
    Conclusion: We shouldn’t be concerned about the wellbeing of rocks. (Value)

    Sam isn’t properly acknowledging P1 as determining the conclusion and it’s not just a case of a hidden premise. He’s claiming that he can determine the conclusion without it.

    By way of analogy, you can’t have a positive conclusion without a one or more positive premises. Imagine if Sam had a syllogism that had one negative premise, one positive premise and a positive conclusion, while failing to state the positive premise and claiming “see, you can go from negative premise to a positive conclusion!” Obviously this is invalid.

    This isn’t trivial, as a lot of criticism of values statements in politics at least requires this kind of nutting out. And because Harris invokes Hume’s is-ought problem himself, it can hardly be treated as trivial.

    Not that I’m accusing Sam of being a social Darwinism (I’m as sick of that being falsely accused as anyone else), but it was a poor appreciation of the is-ought problem that allowed social Darwinism to look respectable up until Moore put it in its place. While Sam may not drive social Darwinism through the gap he’s making in Enlightenment philosophy, that’s not to say that someone else won’t take the opportunity to drive it, or something else similarly nasty, through the gap.

    And again, this is unnecessary. The is-ought problem never prohibited the involvement of science and ethics, and indeed, Moore in his Principia ethica, rather than prohibit it, layed down a basis for this kind of research project!

    My point was that proving is not necessary. We just define good and bad this way.

    I think this statement is at risk of entertaining relativism. Imagine you are part of a culture that eats babies and some anthropologists rock up and ask you why you think this is okay.

    If you answered “We just define good and bad this way”, would you expect them to accept that as a good answer. Maybe they would, noting that in your culture, eating babies is good by definition. I wouldn’t though.

    (The textbook answer isn’t eating babies of course – it’s either throwing them off cliffs in the Mediterranean or killing them in freezing Arctic water).

    We have natural instincts, urges, intuitions, and these can of course lead us to do things that cause ourselves or others to suffer. By definition those actions are wrong.

    Unless you hold to some kind of absolutism (or relativism, which something is say “linguistically true by definition is a certain culture” *shudder*), you can’t “by definition” make any given act wrong.

    You have to justify why a given act is immoral, and hence have criteria to judge acts (in context if you’re not an absolutist). These criteria need to be either self-evident, or derivative of something that is self-evident. Suffering is a pretty good candidate as being self-evident universally (i.e. irrespective of culture), and I don’t dispute this – indeed I share this assumption.

    Following this approach, facts – which are of course essential to the enterprise and hence science is also essential – have a way of making the same act moral in one context but immoral in another.

    Killing out of mercy is not morally equivalent to killing people for fun. Hence while causing suffering may be wrong by definition, killing isn’t. It’s not wrong by definition, it’s wrong when it generates suffering.

    If someone considers science able to inform ethics, then it’s a safe bet they assume that context is important, so to say that science is valuable to ethics while saying certain kinds of acts are by definition wrong, flirts with self-contradiction!

    Currently I am reading Alan Chalmers on them and can see that a straight philosophical application of their ideas to science is too naive. It certainly doesn’t correspond to my understanding of scientific methodology from experience in research.

    Per chance, is it What is this thing called science? you are reading? I haven’t touched it in a while myself. I studied my ethics/critical thinking component under academics from Chalmers’ school at Flinders Uni.

    Fun arguments. πŸ˜€

    I’ll have to get my own copy and re-read it perhaps. It’s been about six years and I know a lot more about Popper and Kuhn now, than I did back then.


  20. I am hoping to soon read the hume work that raises the is/ought issue, as I understand that his actual treatment of this is a lot more sophisticated than the way this is often used.

    My own view on this is that the ought side of the equation is a nonsensical concept. What does ought actually mean as a concept when you unpack it? For me, this smells of human self absorbtion again. We are products of natural systems, and as such, we feel, act and think as those products do. We can, and do, weave our way through the landscape of possible actions and consequences as the entities that we are. Ourselves and our soceities provide the context under which we derive values, and these values can, and do change and develop. To me, all of this lies firmly under the is umbrella, and in fact, there is no ought umbrella. Of course, you can define an ought umbrella into existence just by stating that the ‘values’ that we hold give us the ought, but then, I think you have just defined ought as a subset of is. In which case the whole statement about not deriving an ought from an is becomes nonsensical, because an ought is an is.

    I consider this as being bit similar to free will, in the sense that this also seems to be a concept that is thrown in to grant some sort of special non physical duality to human agency. I personally am quite happy with our agency being described as an emergent property that arises out of the consequnce of complicated physical processes.


  21. Yes, Bruce. I am reading What is this “Thing called Science.” Got on to it from reading his “The scientists Atom and the Philosophers’ Stone” (which I reviewed in From stones to atoms).

    If your bitch about Sam’s presentation is that he didn’t start by acknowledging he was assuming suffering is wrong – then I must conclude that if he had done that you wouldn’t have a criticism?

    Personally this didn’t worry me. It was a given as far as I was concerned and I didn’t feel his presentation lacked anything from not making a firm acknowledgment. Perhaps that acknowledgement is just automatic for me but others need it.

    However, I am a little unhappy that Sam seems to extend this assumption to claiming that morality is about human “flourishing.” Not because it extends beyond considering only suffering (I agree it should), but because the word flourishing could be interpreted in trivial ways. I don’t think, for example, we should be making moral judgements based on “happiness.” It has to be more basic than that.

    I would also be unhappy that anyone had the concept that scientists were going the determine a quantitative measure for “flourishing” and rate societies based on the total sum of “flourishing.”

    You are trivialising my understanding of morality by equating my definition of good and bad based on something like degree of suffering to moral relativism. This is not moral relativism. I would never be in the position of defining something like eating babies as good – obviously.

    In other words, I am not pulling my definition out of the air, or (almost the equivalent) claiming it is given to me by a god. I am basing it on objective facts. On the questions of suffering, and/or flourishing, of sentient, intelligent, empathetic, social species.

    Obviously I can conclude that eating babies is morally wrong. For me that is an intellectual exercise. However, I agree that in another society eating babies may be considered OK. Maybe even encouraged by the prevailing religion. However, here we have the problem that religious morality is not derived from reason. It usually is codified, using supernatural arguments, from evolved moral customs and imposed moral requirements as part of group definition.

    This is where I think Nick has a strong point. perhaps words like right and wrong should not be used. When we consider the law making process these days I think we try to avoid such words. We make decisions on the basis of suffering and harm. That is why we no longer find homosexuality or prostitution illegal – although instinctively many people think they are “wrong.”

    Perhaps, Taking Joshe Greene’s analogy, when we operate in the manual mode words like right and wrong have no use. Maybe they are even misleading as they are really evoking the automatic or emotional reaction we are trying to suppress.

    However, when we operate in the automatic mode we make moral decisions without using logic. We do things because we consider something “right” or “wrong.” This just sums up the intuitions/instincts involved subconsciously.

    Unfortunately, like some other words used automatically, we seem to putting too much meaning on these words. As if “right” and “wrong” had some objective existence or meaning, rather than words defined by real actions.

    The theologians then start using this to justify their god beliefs and we get the idea of divine commands and claims that atheists can never know what is right and wrong.

    In contrast, I think atheists actually have the advantage because they make their moral decisions based on facts (when in manual mode) rather than authority (dictated by religious leaders who know what their god thinks).

    In my view that atheist can then have an objectively based morality whereas the strict theist is stuck with moral relativity – no objective basis and determined purely by what religious leaders wish (hence a moral requirement for tithing).

    In practice of course most of us base our moral decisions (when in manual mode) on reason, logic and objective facts). So most of us can be moral without a god – even though some of us may beleive in a god.

    (I should make the disclaimer that I realise that as individuals we are not very rational and like Haidt’s point that when operating in the manual mode we should attempt to operate in groups, with a plurality of viewpoints, etc.).


  22. Hi Ken,

    I am glad you brought up the manual/automatic mode analogy. I quite liked this, and in particular liked the addition of Haidt’s comment.

    The concept of using a group driven rational process (manual mode) to refine and develop our moral intuitions (automatic mode) sounds like the correct thing to focus on to me.

    Of course, we already have some models and techniques for how to better drive the manual mode part of this, and these, in my opinion are best provided by what has been demonstrated to be effective in the sciences. In other words, empirical, peer reviewed research (and anything else that has been demonstrated to be effective).

    This to me is the key point. I don’t think any amount of mental cogitation alone will get us there.

    It is a bit depressing that these sorts of discussions always seem to get derailed on the one side by philosopher types who think they have some sort of a priori route to nirvana, or religious types on the other side who want/need a rule book. This gets even worse when both of these come together such as with the apologetics crew.

    The bottom line for me is reality. The great thing about times we are now living in is, that although science has not given us all the answers, I think that it is now quite clear that reality can be explained in terms of ‘physical’ processes without resorting to ‘metaphysics’. Advances such as evolutionary theory and modern neuro science have, at least for me, blown the need for any sort of cartesianlike duality out of the water. If somebody wants to hang on to these sorts of concepts, then in my opinion, its up to them to justify them, not rely on old human centric assumptions to smuggle them into every discussion.


  23. One point I forgot to mention above. I am however, still quite interested in the comment from Haidt about virtue ethics, and hope to follow up on this a bit. In particular, the time/effort saving benefits of virtue ethics seem quite attractive to me. I suppose, that you could wrap these fully up into the automatic mode side of the camera analogy, but there is one other point there that jumps out at me.

    My gut feeling about virtue ethics is that these could be defined to fall largely on the side of negative morality in terms of things that you don’t do, and as such, seem to me to incur lower risk.

    You can argue that sometimes inaction can be morally worse than action, but I suspect if you added up all possible inactions vs all possible actions, inaction as an individual would provide the least risk in terms of consequences.

    Given what I have just said, I think I would come out firmly on the side of not pushing the button on both forms of the trolley dilemma discussed in the workshop. For me, you often cannot weight inaction vs action the same when evaluating probable outcomes, as the inherent uncertainty of your prediction about the outcomes needs to increase the weighting for inaction.

    In other words, the less certain you are about the predicted outcomes from the proposed action/inaction, the more you should tend to inaction.


  24. Just picked this news up from HENRY (see Marc Hauser on leave – investigation uncovers scientific misconduct).

    It’s not clear to what extent Hauser is culpable

    “In a letter Hauser wrote this year to some Harvard colleagues, he described the inquiry as painful. The letter, which was shown to the Globe, said that his lab has been under investigation for three years by a Harvard committee, and that evidence of misconduct was found. He alluded to unspecified mistakes and oversights that he had made, and said he will be on leave for the upcoming academic year.”


  25. re Hauser’s leave: I have seen more recent tweets that I believe are referring to the same issue commenting that it may be more an issue of failure to replicate, rather than fraud. I haven’t time to dig them up again, but a heads’ up in the hope it’s useful.


  26. Pingback: A paper by Marc Hauser retracted – Harvard Magazine | Open Parachute

  27. @Bruce, Nick and KenI’ve been follow following the arguments in the comments here. I’m enjoying the debate.

    I haven’t got much to add – you’re all putting the arguments better than I could personally.

    That said, I thought I’d take the opportunity to ask you all a question. I think I’d value your answers.

    Question: What is morality for?

    Obviously I have my own answer to this question. But I’m interested to hear what you guys think.


  28. Daniel, I hope you do participate in the discussions. I had a brief look at your blog and in my view there is some good stuff there. I enjoyed your takedown of Matt Flannegan’s little tirade against the atheist billboards.

    What is morality for, or what role does it play? In one respect morality is innate, part of being the sort of species we are. It’s also there to enable us to relate to each other. Especially when our social groups got larger than kin, tribe or clan.

    That said, of course morality does seem to be wider now. We are more conscious of other animals and their mental lives now. We are also conscious of our environment, even the non living parts.

    But I think our moral systems have an objective basis in the fact that we are a sentient, intelligent, empathetic social species. This means we have evolved with some hard wiring in our brain related to moral feelings, intuitions, instincts, and logic.

    I don’t think it could have been any other way, if we weren’t moral animals I don’t think we could have survived.

    What is your view?


  29. Pingback: A sympathetic take on Marc Hauser and the “scientific misconduct” issue | Open Parachute

  30. Here is an interesting article

    In the comments for this, some people are accusing the author of moral relativism, but I don’t see that myself. I get the impression that the author has no problems saying that these atrocities against women are morally wrong, instead he is questioning the wests actions here. I wonder what Harris would think about this article.

    For me, this article highlights an area where Harris’s argument makes me uncomfortable. I am all for a consequentialist application of scientific techniques to issues of morality, but wonder about the value or application of this.

    Let me explain. In this case, I don’t think the Afghanistan issue requires any further moral judgments. I would expect that there are very few people that would think that the barbarities against women described in the article are morally good, and out of those few people that do, I would expect that almost none of them would change their minds when presented with evidence from the research into human flourishing.

    And now we come to the problem area (I don’t remember his talk too well, but I think that Harris might have referred to this as the 3rd project), and that is, using a science backed consequentialist approach to determining what is the moral course of action. This is where I start to have problems with Harris, as I detect (he doesn’t come out and say it in his talks) that he his expecting some scientific support for some pretty heavy actions, such as military intervention or even perhaps, torture in some circumstances.

    The problem I have with this approach is that, saying something is morally wrong (or even proving it in the manner Harris suggests) is easy compared to saying that any course of action that we would take to try and correct the aforementioned wrong, is morally right.

    The reason for this, is that, in deciding whether a proposed action is morally good or bad, we as good consequentialists, must base these decisions on our expectations about the consequences of our actions. The problem is, that these expectations usually involve highly uncertain predictions about the future. In my opinion, when it comes to decisions such as going to war, it is not sufficient to say “we must take action to stop this moral atrocity” (which incidentally was not actually what was initially said about Afghanistan or Iraq), instead you must also produce a very convincing case that the suggested action is very likely to improve the consequences. In other words, you must have some pretty high confidence levels about you future predictions.

    For this reason, I don’t accept the excuses of people like Tony Blair, or Alistair Campbell that they were acting in good faith to the best of their ability when they went to war. In my opinion, with matters of this level of gravity, they need to take responsibility not only for their decision, but also the consequences of their decisions. When things don’t turn out the way they expected, then it shows that their original decision making process was deeply flawed. I don’t think that any of the principals involved in those decisions to go into war put much thought at all into what the likely consequences would be, instead they felt like taking action and just assumed (against many voices and some pretty clear historical precedents warning otherwise) that western soliders would be greeted with open arms, flowers and songs sung.

    In conclusion. I would expect, due to the complex nature of human societies, that the scale of this problem in many cases is practically unsurmountable. This is why I would imagine that the a project such as Harris is calling for would be primarily useful in terms of providing some form of negative morality, i.e informing us of actions that are bad to take, rather than (positive) actions that are good to take.


  31. I can see your concerns, Nick. However, I think Sam is still pretty vague about conclusions. I was surprised myself at how vague his ideas were when the discussed his moral landscape.

    I agree that the question of intervention is problematic. We can come to agreement about what human rights are or should be but intervention in another country or society is a serious and difficult question and should not be undertaken lightly. Personally I see that these problems must in the end be resolved by the citizens of the country concerned. Massive foreign intervention in South Africa could have created a situation far more dangerous. On the other hand such intervention could be the best solution in some situations.

    And of course there is the political role of sanctions, trade and sporting boycotts etc. As happened with South Africa

    Where these considerations may be more practically real currently is in the problem of immigrant communities. There I think there is more justification for intervention and the arguments of moral relativists need to be countered. Honour killings, genital mutilation etc. Recent decisions of medical bodies to participate in some reduced form of female circumcision are a case in point.

    I suspect the question of judging others moral codes and moral relativism is more of an issue for situations within a country, particularly European countries today.

    But having intervened in Afghanistan, if simply to prevent it being used as a terrorist training base, the US and it’s allies are now bogged down in a civil war where these sort of human rights questions get asked.


  32. I fully agree Ken.

    I think your comment about immigrant communities is very relevant. This seems to be a burning issue in a lot of countries in Europe.

    This can also be quite morally murky however. For example, proposed bans of various forms of islamic headscarves for example, seem to be treading a pretty fine line in terms of the actual vs hoped for consequences.

    Anecdotally, it seems to me that the wearing of headscarves or islamic style beards increased in popularity among people with immigration backgrounds as a response to a perceived attack on their culture/religion. As much as I support the defense of secularity in our communities, I think that such bans could well be both an infringement of human rights, and also counter productive.

    When it comes to things like genital mutilation and honor killings, I don’t myself see any evidence of tolerance for such practices in our societies. As far as I know, such things are highly illegal in most western countries. You could probably find a few post modernist relativists that pop up in the newspaper preaching of tolerance for such things, but I don’t see that there is much (or any) support for that point of view in society at large.

    What I do see however, is how these sorts of things are rolled up together with various forms of “fear of the other” and economic insecurity into popular movements to intensify the pressure on immigrant communities. You can see this in Switzerland with the ban on minarets, and here in Germany there is the so called “Pro KΓΆln” movement trying to ban the construction of a mosque.

    Perhaps in some peoples minds, what I have just written makes me in some way a moral relativist, but I think that this sort of attack is really just a smokescreen to avoiding thinking about and taking responsibility for our own actions.


  33. In my last comment I missed the issue of medical authorities participating in some form of female circumcision. I don’t know anything about this personally, but somewhat ironically I could imagine people making some sort of consequentialist argument here in support of such action.

    I.e. It is better that I perform this surgery because if I don’t it will be performed anyway, illegally by some backstreet operator, with much greater risk to the life of the women. In fact, just this argument has historically been used by the “women’s right to choose” movements.

    I personally would say that there needs to be a blanket ban on things like female circumcision, but I am not sure that I would be able to conclusively “prove” this, on purely consequentialist grounds.


  34. @Ken

    Well… Okay.

    This is a weird feeling. Normally on blog posts like this I have no qualms about charging right in, hacking left and right with a rhetorical blade. But in this case, I’m feeling completely under-qualified. Humility isn’t usually one of my (many) vices.

    On the topic of Sam making people uncomfortable, I’m a little surprised that the most obvious thing about his personality hasn’t been remarked upon.

    Sam Harris is a modern-day mystic.

    He has a history of engagement and practice with some of the more esoteric Buddhist meditation practices. Buddhism (like most things) is a pretty mixed bag. There’s some good stuff in there, along with a whole lot of nonsense. From a few comments I’ve heard Sam make in videos, I’m confident that he rejects the crap. So what’s left over?

    One of the things we can be confident on from modern neuroscience is that the brain is incredibly plastic. It can re-wire itself in new and interesting ways. We also know that training neural pathways has the effect of strengthening those pathways.

    With neural plasticity in mind, consider meditative techniques that require the practitioner to sit for extended periods of time going through a routine mental exercise. Let’s say the exercise is designed to encourage feelings of compassion. Over and over and over again, the practitioner runs through a series of steps designed to evoke an emotional response of compassion, grinding that particular track deep into the brain.

    Extended and regular practice can literally change the brain of the practitioners. If we accept (as I do) that the mind is what the brain does, then changing the brain changes the mind – and even the personality.

    Sam has just such a history. It shows. Little things about him – his turn of phrase, the tranquil calm with which he engages his rhetorical opponents, the soft lilt in his voice, the faraway look in his eyes, the ease and simplicity of his parables* – are all recognizable in other figures (usually religious) that have a similar background. Sam carries himself with the ethos of a Krishnamurti, Dalai Lama or a Zen roshi.

    I think this causes a mixed signal. For most of us free thinkers, mystics are collectively the Them; to be ridiculed and dismissed at best, or actively opposed as the enemy at worst. Not without good reason, of course. Your typical mystic as irrational and unscientific as they come. Much harmful nonsense has been propagated throughout the centuries from the lips of mystics. And let’s not overlook how religious mystics have a habit of rising to the top of the local structures of power, not entirely unlike pond scum.

    Sam – fortunately – seems to have dropped most of the metaphysical framework that has traditionally supported and reinforced meditative practice. So he’s not doing or saying any of the things that cause most mystics to fall into the Them camp. He’s much more of an Us than a Them. But he sounds like one of Them. He has the ethos of a mystic.

    When you take that ethos and then use it to convey the concept that there is an objective standard that can be used to declare a moral statement correct or incorrect – well, that fires off pretty much every red flag we freethinkers have! It combines the ethos of a mystic with the pathos of a preacher! It’s understandable, even inevitable, that he should cause unease in those who he would like to consider his allies.

    Question: What is morality for?

    Answering my own question? How bloody indulgent!

    Interestingly, it doesn’t feel like you actually answered the question yourself. You gave a succinct summary of what the current state of morality is and how it got to be the way it is. Don’t get me wrong – I agree with pretty much everything you said. I just don’t think it exactly answered the question. You’re response was descriptive of whatthe current state of morality is, rather than how the current state of morality performs.

    I’ve always considered that morality is for two things. Well – okay – more than two things. But anything else seems peripheral after these two.

    1) Avoiding the war of all against all

    The first thing to consider is what a world without any moral intuitions would be like. I’ve heard this described as the war of all against all. It would be shit – and as a species, we probably wouldn’t survive long. In some sense, we need morality the same way we need food, water, warmth and shelter. Morality is essential to our survival, both in the innate emotional responses to stimuli, as well as in the abstract philosophical sense. Even once our survival is assured, it’s still essential for our desired lifestyles.

    This comic over at Saturday Morning Breakfast Cereal sums it up for me in a very succinct format. ^_^

    So to one view, if we grant that the war of all against all is bad for everyone, then morality can be viewed as a way of avoiding that outcome by putting into place a code of behavior that will lead to a better outcome for everyone.

    2) Codes of behavior: Do as I tell you!

    This then ties into what I’ve seen as the dominant role of morality in modern society: Controlling the behavior of others.

    In some cases, this is a good thing. For example, the moral arguments and taboos around murder are good things to have in any stable society.

    In other cases, not so much. “The ruler is appointed by God, so do everything he says and give him most of your money and food” is an old one that some places in the world are fortunate enough to look back on with a shudder.

    3) Be like us

    It also seems to me that moral codes can be a kind of signalling of tribal membership. We/Us have this moral code, which makes us better than Them, who we hate for being different to We/Us. Grrr.

    So we can see how this can be valuable in terms of enforcing group solidarity and teamwork. But then again, I don’t think that I need to discuss the many evils of tribalism to this audience. There’s a hefty slice of the bad mixed in with the good.

    I’d go so far as to say that group membership is possibly the worst application of morality, followed closely by the cynical manipulation of the behavior of others.


    I’m sure there’s other things that morality does than just these three roles – but I think these summarize most of what’s relevant.

    The problem for me is that many of the methods to which morality is applied are worthy – but also, many of the methods to which it is applied seem outright undesirable. This is typically where philosophy majors nod sagely and start telling me about meta-ethics, and I get all confused.

    So I fall back on my default position of option 1). Of all the ways that morality is used, I think the most valid lies in avoiding the war of all against all. I think this is where I start to fall into step with Harris. I don’t have the philosophical nuance to poke fine-grained holes in his overall presentation. But I do find his basic assertion that morality is not a neutral term to be very persuasive. I agree with him when he points out that some things are obviously bad ideas. This does seem (to me) to suggest that there must therefore be one or more ways of doing things that will be better (to some degree) than others. And that if we have some metrics to apply – such as the melioration of suffering – we can start to evaluate the outcomes of these different ways of doing things against each other, and make some educated inferences from those measurements.

    It seems to me that Sam is actually leaving the theoretical to the side in favor of the practical. This is where I start to part company with Bruce and the other philosophically erudite critics of Sam’s recent talks on the subject of morality.

    Earlier in the comments, Bruce gave an overview of how he had interpreted one of Sam’s parables. (On an unrelated note, it’s very gratifying to know I’m not the only person who reduces arguments down into logical format when I address their flaws.)

    P1: We should be concerned about wellbeing where there is suffering. (Value)
    P2: Rocks can’t suffer. (Fact)
    Conclusion: We shouldn’t be concerned about the wellbeing of rocks. (Value)

    I think Bruce has this a bit backwards. He’s right of course that the argument above is flawed – I just don’t think that this accurately describes what Sam was actually trying to convey.

    The message I took away from the ‘suffering rock’ parable was something like this:

    P3: We don’t currently have a good consensus as to what attributes a thing must have to qualify for our concern as to its wellbeing.
    P4: However, no-one is concerned about the well-being of rocks.
    C2: Whatever else we may agree is worthy of our concern, rocks don’t have it.
    P5: One of the attributes rocks lack is consciousness.
    p6: Consciousness is one of the things that seems to be a common theme when it comes to our concern for the wellbeing of things.
    Therefore, it seems to be the case that
    C3: We can conclude that consciousness has something to do with wellbeing.

    Which is a bit different. I don’t think that Sam is coming at this argument from a Kantian Pure Reason** perspective. He keeps tying everything he says back to real world observations and examples. I’m under the impression that this is what philosophers call a posteriori** reasoning. (Yes?)

    Sam’s overall approach in the discussion of morality seems to be one disdain (even disinterest) for aloof a priori** reasoning in preference for a reality-based, practical approach as to what makes the lives of sentient beings (humans) better.

    So I think that a lot of the criticism leveled at Sam is misguided. Everyone seems to be acting as if Sam is completely missing that he’s making these philosophical blunders. The problem I find with this criticism is that I think that a) Sam knows perfectly well that he’s making those ‘blunders’, and b) that he doesn’t care, because c) he’s coming at this from the real-world perspective of moral practice as opposed to the abstract world of philosophical theorizing.

    It feels a bit like criticizing Halo because it’s a really bad dungeon crawling RPG. Well – yeah. It’s an FPS, it’s not trying to be an RPG. That doesn’t make the critique wrong: Halo is a really shit dungeon crawler. But the criticism is still weirdly out of context. It doesn’t fit the subject material.

    Yet I’m sure there’s a million things wrong with all that, so I’ll sit back and wait for Bruce (or someone else) to circle them all in red ink and send me on my way. ^_^

    * Note that I use the term ‘parable’ intentionally. Many of Sam’s thought experiments have a parable-ish feel to them. “Imagine a neighbor who believed that he was going to marry Angelina Jolie, and that neighbor made religious justification claim X”. Or: “Imagine an elvis impersonator who thought Elvis was still alive. What if that person made religious justification claim Y for his beliefs?” His thought experiments don’t entirely feel like traditional thought experiments. The tone and emotion is that of a parable. Next time you watch a video of Sam speaking and he gives one of these kinds of examples, analyze it a bit. It’s a really weird sensation when you realize he’s talking in bloody parables and nobody seems to have noticed. ^_^

    ** A couple of times, I’ve used philosophical terminology that I’ve picked up from around the place. I damn well hope I’m using it right.


  35. They have just posted a couple more of the discussion videos on the edge website.

    I picked out two points from those discussions that directly relate to one of my posts earlier.

    1). There was a brief discussion around probability and how this might relate to the trolley game etc… In my opinion, this is critical to both the trolley game an consequentialism as a whole.

    2). Somebody mentioned that there is a very strong action/inaction (or active/passive) difference that shows up in various dilemma games. I.e taking action is evaluated differently from not taking action. I was referring to this earlier as negative/positive morality. What was interesting, is that apparently there is at least one Mayan cultural group that does not differentiate between action/inaction in their moral assessments. One hypothesis was that this is related to the amount of stranger contact apparent in large scale vs small scale cultures, resulting in inaction being judged in a very similar way to action in a society where everybody knows each other and always has. Kinda rings true to me, and puts my earlier comments in an interesting light.


  36. Pingback: A scientific consensus on human morality | Open Parachute

  37. Pingback: Sam Harris on The Daily Show | Open Parachute

Leave a Reply: please be polite to other commenters & no ad hominems.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s