|
 |
October 15, 2003 THE END OF WORK?....I almost forgot, but Marshall Brain (of this website) had an op-ed in the LA Times this morning on a subject that I've been meaning to blog about one of these days: what happens when semi-intelligent robots become available on a wide enough scale that they put lots of people permanently out of work? This isn't a crazy idea, and it's probably not all that far off — a few decades at most. And unlike previous mechanical revolutions, this one will leave the displaced workers with no other industry to go to. What then? I'm probably still going to blog more about this someday, but in the meantime go read the article to get a flavor for the scope of the problem — and the scope of the opportunity. It's not going to be science fiction for much longer. POSTSCRIPT: And by the way, this is one (among many) reasons that I never worry about things like Social Security bankruptcy beyond a 40-50 year time scale. The world will be so fundamentally different by 2050 that it's pretty pointless to speculate that far out. Posted by Kevin Drum at October 15, 2003 10:16 PM | TrackBackComments
Who would have guessed that so many of us could be employed in non-manufacturing and non-agricultural pursuits. I am confident that there will be something for us to do when there are semi-intelligent robots doing many of our jobs. I wouldn't worry until there are fully intelligent robots. (Hey now, keep those snickers to yourselves if you are thinking about the ease of getting a robot intelligent enough to replace me) ;) Posted by: Sebastian Holsclaw at October 15, 2003 10:25 PM | PERMALINKJeez, have you ever seen this guy's "how stuff works" TV segments?
They're the worst crap I've ever seen, they're even full of technical
errors. Star Trek always pretended to have a society without money or the need to work for it. So they went exploring or did what they liked on earth (like being a chef in a restaurant ala Sisko's father). Seemed good to me. Later though, money or straight bartering came back. Damn Ferengi. Posted by: Justene at October 15, 2003 10:41 PM | PERMALINKMeh. It just won't happen on a large scale - at least not much larger than robotic automation is right now. This kinda reminds me of Bill Joy's freakout about nanotechnology, which won't happen either. The reason why nanotechnology won't work that way (that Bill was concerned about) is that current nanotechnology, such viruses or cells, doesn't. Things that small take a long time to achieve their goals simply due to sheer size restrictions. Really have a good look at what's been accomplished with nanotechnology so far, and you'll see it too. Yes, they've built a few really cool little nanomachines, but that's about it. (At least as far as unclassified stuff goes.) We will probably never make nanobots that can create or alter mass materials in any useful or economic way. Robots are limited in much the same way. The best brain on the planet belongs to us. We are not even remotely close to understanding even a portion of the mechanics of it. (We didn't even really figure out the genome as was ballyhooed recently; we only got to the first layer of the messages - kinda like in "Contact" when they figure out the sound from space has pictures of Hitler in it.) And when we do (and if we do) understand enough of it to replicate it, it will open the Pandora's box of ethics that Asimov wrote about in his wonderful robot stories. So, even if we do manage to make something that complex, small, and efficient so as to give mechanical life enough intelligence to walk to work and punch the clock, we may not create it because it will essentially be intelligent life, and as such will fall within our moral and legal rubric. We will do a lot of incredible things with technology in the future,
I'm sure, but intelligent, or even semi-intelligent robots, and useful
nanotechnology will not be part of those incredible things. Well, if everything isn't free, there will be a revolution, much bloodshed and chaos, the contradictions having been heightened. But if everything is free, how will it be allocated. Will our demands expand to absorb the new world of plenty? Some things, like food and music and art, actually have a limit on how much you can enjoy - the download rate into our bodies being finite. But some needs will expand, and with a surplus population idle - I think the newest commodity will be a human slave. Posted by: karl at October 15, 2003 10:52 PM | PERMALINKThis is an interesting exercise in exposing every economic fallacy known to man. Posted by: balz at October 15, 2003 10:54 PM | PERMALINKSebastian, once the anti-U.N. subroutine gets written, the rest of the code anyone can slam out in a few days.;) Posted by: Dark Avenger at October 15, 2003 10:54 PM | PERMALINKWhich is, if it isn't obvious, merely further refinement the social contract as it exists. Posted by: karl at October 15, 2003 10:56 PM | PERMALINKThat's it, the robots will pay for it all. If only there are enough of them to fill in the human spaces. Or illegals under whatever the W plan was for illegals was in 2001. (That was soooooo two years ago.) Posted by: MacMan at October 15, 2003 10:59 PM | PERMALINKSebastian writes: "Who would have guessed that so many of us could be employed in non-manufacturing and non-agricultural pursuits. I am confident that there will be something for us to do when there are semi-intelligent robots doing many of our jobs. I wouldn't worry until there are fully intelligent robots." Semi-intelligent robots could take lots of service jobs, from fast food to dry cleaning to healthcare. One could say that some healthcare positions have already been automated away. Instead of requiring a nurse to take your vitals periodically, there's sensors that will do everything including blood pressure every 15 minutes. And often one of these computers will be provided for every bed at least in the ER. Jobs that require more intelligence are going overseas to cheaper labor markets. Probably what'll be left in the US are things that really are best
done by a mobile, agile, dextrous, fuzzy-thinking primate. Like
construction, repairing things, etc. Digital technology will never produce thought as we know it; its a failed concept. Intelligence requires a brain. Posted by: noam chimpsky at October 15, 2003 11:02 PM | PERMALINKIn the short term, Kevin is committing the "lump of labour" fallacy. Krugman wrote aboutthe same subject in one of his recent Times Op-Eds. However, I don't know if the lump of labour will always continue to be a fallacy. If technological change is extreme enough, even apparently fundamental principles of econonomics/society can change. Krugman (again) in one of his early books gives the esample of how Malthus' insights on the limits of population growth were rendered moot by the Industrial Revolution. Bottom line is : nobody know for sure. Up to now, the truly sentient machine has been like the end of the rainbow; the closer we approach the further away it gets. Eventually it may happen, I suppose. There is the question of what happens if our new artificial minds turn out to be lazy - what if they don't feel like flipping burgers for us? "Here I am, brain the size of a planet and you ask me to supersize that for you. Call that job satisfaction? 'Cos I don't." If almost everybody is permanently unemployed, it will raise a fair amount of existential angst as we try to sort out what to do with ourselves. Playing artist or novelist or even (horrors) freelance pundit is all well and good, but most of the people who try this will find they're not that good at it and eventually give up. Then what? Humans really need some sort of a challenge, or life dissolves into gray meaninglessness. Will we find we have to limit family size? Without labor requirements to restrict it, (robot nannies, anyone?) human reproductvity could easily outstrip the resource base. I guess we could send the robots to build us a colony on the Moon, or on Mars. Or have them harvest resources from the outer solar system. This gets more bizarre the more I think about it. Posted by: jimBOB at October 15, 2003 11:32 PM | PERMALINKRead Kurzweil's Age of Spiritual Machines for a seemingly prescient view of how this can pan out. Boggles the mind once you think about it, which proves your point, Kevin. Our future world is at the mercy of the fundamental (and unpredictable) forces of entropy and exponentially accelerated hyper-evolution. No sense in worrying about it, cuz whaddya gonna do? Posted by: nova silverpill at October 15, 2003 11:35 PM | PERMALINKjimBOB writes: " Humans really need some sort of a challenge, or life dissolves into gray meaninglessness." The more important question is, how are they going to make a living and keep food on the table? Intelligent robots doesn't mean you get free stuff. For that you need some kind of nanotech fabrication capability that can use junk and garbage as raw materials to make new things, atom by atom. What you're talking about is a good description of the setting of Iain Banks' novels. Posted by: Jon H at October 15, 2003 11:45 PM | PERMALINKJeez, folks, enough with lump of labor fallacy. This has nothing to do with that. Sure, it might turn out that AI never happens. But on the assumption that it does, then AI will do *all* the work cheaper than humans. We'll continue to make more and more stuff, but it will all be made by our friendly robots. Come on, give this a little thought. Remember, economic theories have changed throughout history because circumstances have changed. Economists of the 18th century didn't study unemployment because there was no such thing. This is such a change. If it happens, current economic theories will be thrown out the window. Posted by: Kevin Drum at October 15, 2003 11:46 PM | PERMALINKFor that you need some kind of nanotech fabrication capability that can use junk and garbage as raw materials to make new things, atom by atom. I believe this is called "farming." And yes, sentient robots ought to be able to handle it. Posted by: jimBOB at October 15, 2003 11:51 PM | PERMALINKYahmdallah writes: ". The reason why nanotechnology won't work that way (that Bill was concerned about) is that current nanotechnology, such viruses or cells, doesn't. " Uh, current nuclear fusion doesn't vaporize cities, it operates harmlessly in stars. But that doesn't mean that an H-bomb wouldn't do a job on Hackensack. Posted by: Jon H at October 15, 2003 11:55 PM | PERMALINK"I believe this is called "farming." And yes, sentient robots ought to be able to handle it." Labor isn't the only expense involved in farming. There's also irrigation, fertilizer, pesticides, seeds, etc. And someone has to pay for the materials used in repairing the robots, or for the cost of replacing worn-out robots. Posted by: Jon H at October 15, 2003 11:58 PM | PERMALINKJust how intelligent does a robot have to be to flip burgers and respond to a customer-made order? Not nearly so intelligent as to be considered sentient. Just has to know what it's doing with regard to ensuring the food gets cooked. Just has to recognize the difference between a spatula and a burger patty, and such. I'm confident in my lifetime we'll have machines that can do that. There is an awful lot of mindless work out there, and machines will eventually be able to do most of it. Now, what jobs replace them? Almost useless to speculate. I know this much: something has to be done with people. We can't throw them on the refuse heap just because rich people own the robots doing all the old jobs. They'll have to have something to do. Necessity is the mother of invention, so I believe we'll find something for them to do. There might still be some steps backward and some real changes in society and the world in that time. The aggregation of wealth into fewer and fewer hands is unsustainable. Hopefully that's a societal problem that gets solved peacefully rather than exacerbated until the revolution stage. Posted by: Norman at October 16, 2003 12:05 AM | PERMALINKDigital technology will never produce thought as we know it; its a failed concept. Intelligence requires a brain. Prepared to wager on that? ;) I should add that I happen to agree that the architecture involved in creating "sentience" can't be efficiently simulated on anything less than a superQC, but I'm also firmly a believer that human consciousness is "computable" a la Church-Turing. Posted by: Anarch at October 16, 2003 12:13 AM | PERMALINKDigital technology will never produce thought as we know it; its a failed concept. Intelligence requires a brain. Unless you mean our specific, current attempt at computing, I can't agree with that. You either accept one of these two: 1) The brain is just a chunk of matter, and it is a spirit given from god that makes us human. 2) Or the brain is just a machine, in which case one day we could build one. If you want to say that it is 1), OK. If not, than you have to go with 2. I am not sure what you mean by 'brain' but it sounds like you are leaning towards 2 with that word. Posted by: Timothy Klein at October 16, 2003 12:42 AM | PERMALINKThe good news is that any job a machine can do isn't a good occupation. We're still in the "what is wealth?" argument I think, but if we go with the idea that wealth is the product of labor then we're going to have a much more wealthy society. What are our needs? Food. Shelter. Health Care. Transportation. If these are in super-abundance, then life becomes a more interesting market for entertainment and leisure pursuits, at least where real estate is either available and/or undesirable. You may be able to live a life of slack in Fresno, but if you want to run with the bulls in San Francisco you'll still have to work your ass off to out-earn people who also want to live there -- as we saw during the dotcom days, when everyone is a millionaire rents and home prices will simply rise to meet the market. There are social darwinism arguments in play here too, where undesirable economic activity may be encouraged by unrationed necessities of life. But in general there will still be a market (and thus monetary value) for everything that is not machine-made, like say personalized health care, plays, concerts, and other artistic pursuits. Sounds like an interesting world to me... Posted by: Troy at October 16, 2003 12:47 AM | PERMALINKOh yeah, expect athletics. Hell, the more I expound on this, the more this Brave New World looks like ancient athens, where the slaves did all the work. Posted by: Troy at October 16, 2003 12:51 AM | PERMALINKRemember what Sam Waterson said: Robots are everywhere, and they eat old people's medicine for fuel. http://www.robotcombat.com/video_oldglory_hi.html Posted by: Anonymous Blogger at October 16, 2003 12:51 AM | PERMALINKThe robotic revolution may begin even sooner than Marshall Brain's
date of 2040, because we don't need human-level sentience for many of
the most basic jobs. Bees can harvest honey, communicate with each
other, and build rather complex homes. How far off are robots of
bee-level intelligence? (and of course the dystopia would be worldwide war 24/7. Wars make the best reality shows, provide full employment, and the best way to remove super-abundance of wealth is to blow it up). Posted by: Troy at October 16, 2003 12:55 AM | PERMALINKHow far off are robots of bee-level intelligence 10 years ago? The main challenge with robotics is machine perception. This is a very hard problem, but with massive computer resources (eg. virtual paralellization, brute-force pattern matching) looks solvable. Like being able to distinguish a flower from the background image. But we've all had bees buzz our hawaiian shirts, so obviously their algorithm sucks too. Posted by: Troy at October 16, 2003 12:58 AM | PERMALINKAll I can say is that I am an automation engineer, a villain straight out of Vonnegut's Player Piano in that I have spent my entire career getting paid well to replace human labor with computer-controlled electro-mechanical devices. On the one hand I am relieving humans from doing dull, repetitive, and mindless tasks; on the other I am forcing people out of jobs. As an engineer I feel good about what I do; as a liberal, maybe a bit guilty sometimes. Posted by: mat at October 16, 2003 01:11 AM | PERMALINKYou know, one very interesting possibility for robots, to my mind, is warfare. The current crop of UAVs that we use in the US are just the tip of the iceberg. This could be a nice development: "The robot armies of the US and China fought again today, with heavy machine losses on both sides. No humans died." Or it could be a very nasty devlopment: "Terminator hunter-killer robots roamed through the streats of LA today, killing tens of thousands of civilians ...." I think we are actually a lot closer to a massively automated society than we all realize. We don't need HAL 9000 sentient super computers for this to happen -- we just need specialized, moron machines that can do one thing very well. Posted by: Timothy Klein at October 16, 2003 01:18 AM | PERMALINKRobot armies exemplify Einstein's comment: "Everything has changed except our way of thinking." In America, we're already close to that point. We don't call our workers robots [1], though, we call them Mexicans. Reinventing humans is a cool endeavor, but so is reinventing the wheel, which is why engineers enjoy their work. Why would we think that an automated workforce would work to our benefit? Wouldn't our masters continue to insist that their profits should be purely return on investment and free of taxes, when this is already what they're doing? People - members of the human species - are REALLY smart, and, sadly, pretty cheap, and likely to remain so for a while. Your next shoe will likely not be made by a robot. [1] Karl Capek: the play "RUR", the novel "The War of the Newts." "Robot" is from the Slavic verb for work. Posted by: bad Jim at October 16, 2003 01:37 AM | PERMALINKYes, bad jim is right -- it's tough to beat the $45/mo wages a shoe factory worker makes in Indonesia. I was looking further out though, where the labor robots are smart and capable enough to make themselves without human labor. This could be a Vernor Vinge "singularity" event (what progresses from it is totally disjoint from what preceeded it). Posted by: Troy at October 16, 2003 02:29 AM | PERMALINKThere was some speculation a while ago, before the Internet Bubble popped, that we were entering a new phase of macroeconomic structure: the end of the business cycle and all that jazz... But the real structural change is happening as we speak. The mass replacement of workers by robotics and more importantly international "outsourcing" is the tsunami that is washing over the American economy as we speak. How can consumers buy things without jobs? When these emerging facts on the ground hit up against the bedrock of market ideology in the American psyche--I don't think it's going to be pretty. The government isn't going to be handing out $25,000 to folks anytime soon--it's against the rules of a market economy. But market forces seem destined to evolve into ultimately providing few opportunities for well-payed jobs for the masses--which means a destruction of markets because the consumers just aren't there. The "supple-siders" have owned the public debate for decades now. But I think "demand-side" economics and theory is going to define our coming century. Somehow, the rules are going to have to change. Further reading on this subject: The End of Work I'd love to hear of other links that address these issues. Posted by: Tim B. at October 16, 2003 02:38 AM | PERMALINKThis is not exactly a new "problem", and it's much ado about nothing. My dad was a postal worker until he died in 1965. Back then, all the postal workers thought the new 5 digit zip code (which replaced the 2 digit zone which came between city and state in an address) would cost a lot of them their jobs. And isn't anyone else reading this website old enough to remember that, in the mid seventies, many people were saying that computers would one day do a lot of jobs that previously were done by humans? It's pretty silly to get worked up about it. The concept has been around at least since Henry Ford put the buggy whip makers out of work. Posted by: Robert at October 16, 2003 03:14 AM | PERMALINKRobert, so computers don't do lots of jobs that were once done by humans? Basically you're coming up against the inductive gap, you're supposing that because former predictions of the end of work have been incorrect(because these predictions did not accurately predict new jobs created by new opportunities) then future predictions of the end of work will also prove incorrect, but unless you actually can point to new jobs that will be created by the opportunities opened up by robots taking the old then the argument is worthless. In the past few generations many jobs created have been service jobs, McDonalds is currently experimenting with automated cashiers, probably this will mean that at least some people from McDonalds will be fired when and if they actually computerize the ordering process. Those that remain will probably be somewhat higher paid because they will probably learn to work with these machines. Nonetheless as the need for service jobs gets further eroded by robotics this will create a problem for the economy in that if you have a good job and lose it today, you can always drop down into a crap job if you actually need to. The technological innovations of past decades have generally replaced what used to be thought of as good jobs, maybe boring jobs but high-paying, now they are replacing lousy jobs. One of the traditional problems with lousy jobs is that they have been sort of hard to fill, and have a high turnover, if that is no longer the case then a safety net is removed. This is just a first thought on the matter, I haven't given it any great consideration. Posted by: bryan at October 16, 2003 03:34 AM | PERMALINKWow that is much more inspiring than "Bush lied Plame died" stuff. Just my two kopecks of wisdom: A.I. is definitely coming to the workplace near you and conseqences
will be very different from laying off buggy whip makers. From the
moment machine acquires sentience it's race against time to My take on this - If we won't solve the problem of transfering human intelligence into artificial brain before it will start evolving on its own -we'll go the way of Neandertals. And all in our lifetime. Pity you guys failed with this Gore/Bush thing in 2000 Posted by: Russian at October 16, 2003 03:40 AM | PERMALINKRemove the income taxability cap of 88,000 dollars and tax all income above 10 or 20 thousand dollars for Social Security and Medicare. INSTEAD OF COMPLAINING ABOUT REPUBLICANS. PUNISH REPUBLICANS. What an idiot. Hey, knucklehead, in case you didn't read the letter from Gore, it said nothing positive in response to your coin...just a thank you for sending it to him. And in case you are too stupid to figure out why Gore sent you the letter...no, it wasn't because he liked the coin and agreed with it. He went on to hit you up for cash for his PAC. Hey...did you send him any? And by the way...if the left boycotts Disney, where will Gay Day be? You may not like Bush and the Republicans, but to say Arnold stole the election is idiotic. Are you typical of the Democrats? Posted by: Robert at October 16, 2003 04:47 AM | PERMALINKBryan...someone will have to service the robots. Someone will have to manufacture them, market them, ship them, service them. Yeah, one McDonalds may end up with fewer employees, but at the rate they are expanding, I don't think even that will be a loss of jobs...just moving jobs from one location to another. Wal-Mart in my community is a perfect example. Sure, they've gone to self-scan checkout lines. One employee for six scanners, instead of six employees. But they also have opened up a new store in town. And most people will always prefer to have a 68 year old grandma saying "Hello. Welcome to Wal-mart" instead of a robot. Posted by: Robert at October 16, 2003 04:57 AM | PERMALINKRobots can't be taxed. And, if BIG companies don't need lots of semi-skilled workers for routine tasks, then the poorest of the poor will have to spread out and live hand to mouth, hard scrapple lives, off the beaten path. Because without incomes there are limitations on movement. And, there will be an underground economy. By definition, not taxed. The other reality is that fuels are a consumable commodity. Need to be carried over long distances. And, if 'robotics' get used, let's say at sea, in the movement of great container ships, etc., wouldn't these things be sitting ducks for pirates? Oh, and if the fuel gets to be too expensive, where capitalisms rewards are deminished, then ever increasing costs just squeeze the labor market all the more. OF COURSE, when the Black Death swept through Europe (3 waves), beginning around 1348, the loss of life was so astronomical that FOR THE FIRST TIME, labor actually began to be worth something. It was slow going (since the Church fought like crazy against the peasants improving their lives with better wages) ... but as long as there's an imbalance between population and workers, fields lay fallow. People leave towns and fields to die. (Well, in the Black Death example they were already dead, here.) And, the constant wars of the European aristocracy became extremely savage. Robots can't fill the gap if they require MORE fuel that may be the commodity that goes into the shortest supply? In a complex world you're inviting catastrophe when you move a single domino. Leisure time will be an oxymoron. Posted by: Carol in California at October 16, 2003 05:04 AM | PERMALINKHeh, Kurt Vonnegut spoke to Kevin D.'s question over 50 years ago in his first novel, Player Piano (1952). I think the following review of the book might cheer Kevin up as to his future prospects: http://www.vonnegutweb.com/playerpiano/pp_nytimes.html The Engineers Take Over - by Granville Hicks (NY Times, June 3rd, 1963) Two books that were popular several decades ago--Ignatius Donnelly's Caesar's Column and Jack London's The Iron Heel--are brought to mind by Kurt Vonnegut's novel. In it, as in them, we are taken into the future and shown an America ruled by a tiny oligarchy, and here too there is a revolt that fails. The important difference lies in the fact that Mr. Vonnegut's oligarchs are not capitalists but engineers. In the future as he envisages it, the machines have completed their triumph, dispossessing not only the manual laborers but the white collar workers as well. Consequently the carefully selected, highly trained individuals who design and control the machines are the only people who have anything to do. Other people, the great majority, can either go into the Reconstruction and Reclamation Corps, which is devoted to boondoggling, or join the army, which has no real function in a machine-dominated world-society. ... Posted by: David W. at October 16, 2003 06:05 AM | PERMALINKThis interview excerpt with Vonnegut back in 1973 is a fun read too: Vonnegut on Technology & Cheesy Little Religions http://www.vonnegutweb.com/playerpiano/pp_scholesnterview.html Posted by: David W. at October 16, 2003 06:10 AM | PERMALINKThis really is THE issue in western society for the 21st centry. Tim B. is right. Keeping our economies afloat is going to involve moving to more Demand-side economics. Especially keeping the service economies alive, considering that is where the hiring is...and it's also the most vulnurable sector in a recession. It's going to require courage, balls, and the smarts to completely turn the corner. If it's not done when it needs to be done.... Game over. Posted by: Karmakin at October 16, 2003 06:12 AM | PERMALINKA point - I was talking to a customer of mine the other day about the movement of jobs offshore, and (being a conservative economist in SoCal) his view was this: If htere is a movement away from manufacturing jobs, its no different than the movement away from farming jobs in the first industrial revolution. Back then, farmers across the country lost their jobs because industry built machines that made farming far more efficient than it was in the past. But the people in farming realized the trend, and moved to the city to join the new neccesity for manufacturing jobs there. Thus the first industrial revolution was created. That's why we need to not worry about all of these manufacturing jobs moving off-shore (and to robots) simply because this is the second industrial revolution. There is one little problem I see with this. If, in the first industrial revolution we moved from one type of job to another, what - in the second industrial revolution - is the replacement for our jobs? Retail? Repair? Realistically, these are not acceptable options. Retail, pioneered in the modern sense by Wal Mart, is a slave wage that does not offer the great majority of workers anything that was offered under the manufacturing sector of the ecenomy. Repair is also not an option. If robots take over the jobs that people do right now, then those people will still lose the benefits given to them under the current manufacturing-based economy that we work under. This will create a severe problem in our medical sector. What do doctors do? Do they treat a majority of patients that do not have healthcare due to their hypocratic oath, or do they turn them away because they don't have healthcare? The fact is that with this many people living 'off the system,' the system will fall. Even with the imminent collapse of Social Security and Medicare, these people will still break bones, still need their appendix out, still need their tonsils out, and still need all sorts of procedures that will end up on the backs of the people who are paying for these services. So the problem is, if you think this move to robotic labor is good, look at the overall consequences of it for society. Its not good at all. Posted by: Jesse in SD at October 16, 2003 06:23 AM | PERMALINKBrian Stableford wrote a great sci-fi novel that touched on this,
called Journey to the Center. He describes a society where they've
solved the problems of energy supply by tapping into a virtually
limitless resource that's called "the frame force". Whatever. I think we're in this process right now, it's just slow. Automation is replacing workers as we speak, but unless some real form of AI can be invented - or at least until you can program a computer to make fewer mistakes than a human - it's really just the old process of industrialization. Look back to the days of waterwheel powered mechanical spinning and looms, it's the same issues. Posted by: rhinoman at October 16, 2003 06:56 AM | PERMALINKMr. Drum has glimpsed the coming of the Singularity:
Bad Jim -- Robot armies exemplify Einstein's comment: "Everything has changed except our way of thinking." You and Al are exactly right. There is no alchemy in any of this -- nothing creative whatsoever. The desperately disguised morbitity of viagra-hopped up onanists. Posted by: cs at October 16, 2003 07:15 AM | PERMALINKIn the future, when robots will be doing all the work, what will humans do? Why, blog, of course. Posted by: tristero at October 16, 2003 07:40 AM | PERMALINK"There was some speculation a while ago, before the Internet Bubble
popped, that we were entering a new phase of macroeconomic structure:
the end of the business cycle and all that jazz... Simple. We put the excess people to work designing and making things that no one's ever seen before. There'll never be a shortage of new products and services that people will want. If everyone in the third world knows how to crank out software, we start making personal aircraft and spacecraft, taking advantage of that cheaper software to help the operators control the things. Once we know how to make spacecraft dirt cheap, we start building housing in space (self-contained permanent habitats). Then we make better ones. Then still better ones. Then interstellar spacecraft. Somewhere in there, we make anti-aging pills. Then better anti-aging pills. Then even better habitats. Then anti-matter. (Then everyone scatters like crazy lest a lone nut exterminate the entire human race, and demand for even better spacecraft shoots up). Then we build planets. Then Ringworlds. (But not too heavily populated, the more people on your planet, the higher the odds someone will blow it up.) Not to mention all sorts of little things along the way. Some of our laid-off programmers will end up making a robot porter that follows you and carries your stuff, so you can walk around in a city without bothering with a car and without worrying about cargo. We'll always have something useful to do for as many people as want to do it. As long as we let people dream up new stuff and try to sell it, without making them play Mother May I with bureaucrats (without which, I'd bet we'd have that anti-aging pill right now), we'll never lack for useful things for people to work on. Posted by: Ken at October 16, 2003 07:51 AM | PERMALINKEveryone who has just ploughed the back-forty while the wife spun some cloth on a hand loom raise your hand....no ? I think we'll survive, Luddite worries and science fiction notwithstanding. Posted by: mark safranski at October 16, 2003 07:57 AM | PERMALINKThis conversation is both educational and riotously funny. Most of
you seem to work for a living -how do you find the time to do this if
you're at work? Or are you just staying up all night? I see lots of
posts at 3 a.m. and they're not all from Russian. I realize all the
posts are on CA time, but still. We can all work for government, explaining to each other how patheticly we manage ourselves. When robots displace almost all of the workers, who will buy the goods and services that the robots produce? There can't be any profits if no one is able to purchase anything. In the end-case scenario, won't the market economy just collapse? Posted by: keefer at October 16, 2003 08:27 AM | PERMALINKHell, no, Granmere. Kevin- You raise a very good question. One I have probably considered too often. Although I often disagree with you(as I am a strong conservative) one thing I have always thought is that once labor no longer has a variable cost component (and a low enough fixed cost component) that it is more cost-beneficial than human labor, government and social services are in for a huge sea change. In some ways, I think Marx's Feudalism->Capitalism->Socialism-> Communism theory may work. I've just always figured Capitalism works it way to socialism through a much slower/kinder/gentler method...increasing widespread ownership of companies (middle class investors jumping into the stock market). Here's the key eventually, we all become owners of the companies that produce and live off the dividends of the shares we own. Crazy idea yeah, but then again who really knows...Eventually inequalities decline or rise based on other factors (you're right goods would still carry a cost) who gets the best deals, whether one provides some goods for themself (gardening or personal agriculture.) Of course if that doesn't happen (we aren't peaceful about it) we start along man's favorite pastime...killing each other. At the same time we'll see how long it times automatons to perform well enough if ever... Posted by: Bill S. at October 16, 2003 08:31 AM | PERMALINKTotal automation would be a nightmare. A couple of things First, our rate of reproduction is dropping, i.e. decreasing labor pool, this is even more pronounced in Europe. Slowly replacing human labor with machine labor may have no impact on productivity. Second, companies put in lots of computers to make things easier for the people, but does the accounting department where y'all work use fewer people than it did before? (I picked on accounting because it is usually the first department computerized). Current mechanization needs people to build, install and maintain it. Does the need for people really drop? Or did they shift jobs? Posted by: Ron at October 16, 2003 08:56 AM | PERMALINKWe will only have a "problem" when computers are smart enough and cheap enough that they can replace all of the tasks we would assign to dumb people. Then those dumb people will have no jobs. (I suspect that in that event stupidity will become a disability, and we will provide for them through some sort of ADA/Soc. Sec. Disability program. The SS failures that are coming up don't apply in that scenario - if we can produce basic goods with machines, scarcity as we know it becomes a less pressing concern.) Markets won't collapse, but prices of goods should - the reason we would use robots is if they can produce at very low cost. A very low marginal cost of production implies a very low price in a free market. Value in goods will continue to transition to "service" value. Today, we live in a society where people expect to be able to get service 24 hours a day, 7 days a week. IBM expects to call its suppliers, and talk to the account manager, no matter what, no matter when. We expect to be able to drop in a the grocery store at 2 in the morning, the ATM at 4. In future we will see that trend accelerating, and we will see most people working in the following areas: "Individualized" production of hand made goods. When machine made goods get really cheap, the mark of wealth and "quality" will be hand manufactured goods. Albequerque or Pike Street Market on a grand scale. "Personalized" service. We get this today with "boutique" shopping, home delivery of pizza, things like that. Machines could have replaced people in restaurants and fast food in the '50s, had people been willing to go that route - cooking a burger really isn't that hard, even for a machine. (Your frozen meal in a box at the grocery store was, after all, made by a machine, and has been for decades.) Computers could make your burger to order, the way you want it, TODAY. People just won't shop at that store - they like "service", even from surly teenagers. We COULD clean our own clothes, do our own taxes, cook our own food, etc. Some of us still do. More and more of us won't. The physical components of a meal will be very cheap, but the service is where the value is - compare a restaurant meal to buying the same food at the grocery store and cooking it yourself. Don't worry, we'll find jobs for everyone except the true morons. We can probably support THEM. Posted by: rvman at October 16, 2003 09:09 AM | PERMALINKA lot of people are really missing the point here (although many are not). The End of Work I believe that there is a non-fiction book out by that title. Anyway, I suspect that one of the first things to go in the not too distant future will be the legal ablility of both the husband and wife to work. It simply won't be allowed. Why? The environment will not the only victim burgeoning, out-of-control population. There simply won't be enough work to go around. A little farther down the road with no thought to population control means that ever growing pecentages of society are unemployed. 10%, 20%, 30%, 40%, 50% ... Unless something radical happens this will mean that poverty, homelessness and all the attendant proplems will spread. Posted by: a at October 16, 2003 09:30 AM | PERMALINKRon writes: "Second, companies put in lots of computers to make things easier for the people, but does the accounting department where y'all work use fewer people than it did before? (I picked on accounting because it is usually the first department computerized). Current mechanization needs people to build, install and maintain it. Does the need for people really drop? Or did they shift jobs?" Accounting jobs are starting to go overseas. And the building and maintaining of the software for business is also going overseas. The offshoring is the first part of the problem, which will be compounded by automation. Sending service jobs offshore will put "knowledge" workers into lower-paid service jobs, at first, but then AI robotics will take up the lower-paid service jobs. So then you'll have technology removing the lower-paid jobs, and globalization removing the higher-paid jobs. Any new companies and fields created will also be operating under the same market conditions, so are likely to opt for offshore workers and robots rather than paying high wages to Americans. Even if some out of work engineers develop flying cars, they are likely to adopt the same business practices that put them out of work in the first place. End result: even if new technologies, companies, and industries arise, they won't produce many new jobs. At least not in developed countries. And certainly not enough in developed countries to employ everyone who lost their jobs to automation and offshoring. Posted by: Jon H at October 16, 2003 09:32 AM | PERMALINKrvman writes: "We expect to be able to drop in a the grocery store at 2 in the morning" That probably just makes business sense. The store is typically already full of people restocking for much of the night. The A/C and refrigeration, a significant expense, are still running. All that's missing is someone to work the register. Grocery store profit margins are razor thin as it is. If you can get some sales in overnight, at the price of a cheap checkout person, why not? Posted by: Jon H at October 16, 2003 09:40 AM | PERMALINKUh, yea. Bob Wilson laid all of this out 25 years ago. Posted by: Harry Tuttle at October 16, 2003 09:41 AM | PERMALINKI've heard the death of work meme before...yawn. Contrary to one poster above, employment is rapidly expanding in medicine. Biology has become a computational science, and there are huges needs for highly educated mathematicians and computer analysts to interpret such data. I don't think the need to educate people is going to disappear either--it will only continue to expand. What is already highly apparent, and indeed has been occurring for some time, is that poorly educated people in the developed world are increasingly being left behind in the astounding prosperity of the developed world, and well-educated people from the developing world have greatly increased opportunities, due to the premium put on knowledge workers. These trends are likely to continue in the short and medium term, and as Kevin says, who knows what the long term will bring? Besides, our economy is already very substantially a service economy already. Do we really need nail salons and pet clothing boutiques? Yet these are viable niche industries nonetheless. In California, being a psychic friend can even be a viable way of making a living! Our resources are so abundant that we are already in a "post-modern" economy in substantial ways. Posted by: Daniel Calto at October 16, 2003 09:45 AM | PERMALINK"Any new companies and fields created will also be operating under the same market conditions, so are likely to opt for offshore workers and robots rather than paying high wages to Americans. Even if some out of work engineers develop flying cars, they are likely to adopt the same business practices that put them out of work in the first place." But it takes time for foreigners to learn how to make our newest stuff. It takes even more time to teach a robot how to make our newest stuff. And when they do, we go on to even newer stuff. Repeat until we rule the Universe. That should keep us busy for quite a while. Posted by: Ken at October 16, 2003 09:59 AM | PERMALINKTo steal an idea from the Stainless Steel Rat books, no need for robots per se - I'm sure a fully-automated McDonald's is technologically feasible. Just have a guy roll up once a day to load food supplies in the back and take money from the front, let machinery in between do all the work. Not that different from a vending machine. Yes parts would wear out and break, you'd have to have mechanics making the rounds, but you could get rid of a passel of teenagers and poorly-paid managers. Uh, I'm not advocating this, just saying it's feasible. Posted by: Librul at October 16, 2003 10:00 AM | PERMALINKgoody! a thread where everyone can have an opinion and not start a flame war.. Ken writes: "But it takes time for foreigners to learn how to make our newest stuff." Not that long, especially when our colleges and grad schools are full of foreign students. It's not like they're going from stone-grinding maize to fabricating integrated circuits. They're going from fabricating older integrated circuits to fabricating the new one. There are already American companies that consist of a very small US management office, with everything else done overseas. "It takes even more time to teach a robot how to make our newest stuff." Nah, I doubt it. Once robots get to that point, they'll probably learn a lot faster than people, because you can't download a program into a person. Also, teaching the second person takes the same amount of time as teaching the first. With robots, teaching the first might take a while, but after that it's a matter of copying data to new robots. "And when they do, we go on to even newer stuff." Technological advances don't come as often as you think. Posted by: Jon H at October 16, 2003 10:18 AM | PERMALINKA couple other people hit on this but the important thing to think about is how much recognition, decision and processing power does it take to do certain tasks? Not much. There's no need to ever achieve a robot that can "think" like we can, "thinking" isn't required for... well for all intents and purposes, it's not required for any of the work we do if you boil it down to just the process and product. Take something that is purely a product of human imagination, like filmmaking. Surely you'd have to be able to think to make a movie, right? No way. There are rules for every aspect, scriptwriting (a computer properly programmed could make a script certainly as good as Charlies Angels), the technical aspects of production, composition, editing, etc. Comic timing could be approximated though algorithms, proper mic and light placement would be a snap, editing could be accomplished... Imagine a studio that is nothing but green screen and fully automated. Shot #15 is a wide master taking place in full sunlight? Up goes the auto HMIs, a voice says "action" and the real actors do their bit. Once the correct phrases are spoken there's a little trail and then "cut". Voices are processed and evaluated for the correct emotional cues... another take. Hell, in 20 years you'll be able to render humans so perfectly through cgi you won't need real actors or anything. Now, that doesn't mean any of this will happen, or anyone will want it to happen, but it sure as hell could happen and, given the public's taste in films, I don't see why virtual movies, made completely by computer, won't happen. We like shitty movies now, why wouldn't we like the same crap only not made by human hands? The point is a computer does not require intelligence, thought, or self-awareness to be able to replicate most anything we can do. Humans run on emotion and we tend to think of the things we do in terms of struggle or determination or imagination, etc. And in fact all the things we do are products of our creativity and imagination. But, once something is done it can be copied. Once enough original things are done rules for approximation can be distilled. Certainly, properly programed, any computer could approximate just about any sort of work of art or other product of man- and it wouldn't have to be intelligent to do so. Sure, man will probably always be the font for original things- but how muc originality is in your daily life? Huh? The food you eat, the cars you drive, the TV you watch the chores you do- it's nor picasso, it's rote crap. We most definitely are heading to a time when anything we do in our daily lives a machine can do cheaper and better- and if we stay in the same economic/social model we're in it's not going to be very pleasant because captains of industry care about bottom lines and their profits, and that's about it. Posted by: Tim at October 16, 2003 10:22 AM | PERMALINK"Technological advances don't come as often as you think." They do when you remove government controls on what's permitted to be sold, who's permitted to sell it, who's permitted to buy it, and so on. There's a reason that the products of the computer industry have been exhibiting phenomenal improvements in price and performance for years. "Not that long, especially when our colleges and grad schools are full of foreign students." They've been full of foreign students for quite a long time now, and we were still able to dominate the software industry for years. "Nah, I doubt it. Once robots get to that point, they'll probably learn a lot faster than people, because you can't download a program into a person." Until robots can invent on their own, someone still has to break down the production process into steps a robot can understand. That's likely to remain non-trivial for quite a while. Teaching smart people to do something new is comparatively easy. Posted by: Ken at October 16, 2003 10:23 AM | PERMALINKDaniel Calto writes: "Contrary to one poster above, employment is rapidly expanding in medicine. Biology has become a computational science, and there are huges needs for highly educated mathematicians and computer analysts to interpret such data. I don't think the need to educate people is going to disappear either--it will only continue to expand. " So why won't this work be done overseas, where it's cheaper? Data is easy to ship over the wire. Might as well have the mathematicians and computer analysts be in India or Pakistan, making $6,000 a year. Educate people for what? The smart way to go, it seems, is for school kids to go with
vocational school and forget college, except maybe some community
college business and accounting courses. Enough to, say, learn how to
weld and start a welding business. Uh, folks... Computer don't learn. They're programmed. Seems like splitting hairs but it's actually a huuuuuuuuuuuge difference. Ken writes: "They've been full of foreign students for quite a long time now, and we were still able to dominate the software industry for years." And now we've got record unemployment in the US software industry, and booming demand in India, and large companies setting up large facilities in India and China. Things change. Ken writes: "Until robots can invent on their own, someone still has to break down the production process into steps a robot can understand. That's likely to remain non-trivial for quite a while." I was thinking of 'making things' as 'manufacturing', not invention. Inventors are, by their nature, very few in number. And certainly not numerous enough to make any significant dent in unemployment. Wal-Mart employees replaced by robots aren't going to become inventors as a fallback. In any case, the inventors can be replaced with offshore workers who are cheap. And this is being done, by companies like Intel, who are building large facilities in Russia, China, India, and elsewhere. So even the few inventors aren't safe from being replaced. Just not with robots. Posted by: Jon H at October 16, 2003 10:38 AM | PERMALINKTim writes: "Computer don't learn. They're programmed. Seems like splitting hairs but it's actually a huuuuuuuuuuuge difference." Er, computers can learn. Via neural networks, for example. Posted by: Jon H at October 16, 2003 10:42 AM | PERMALINKJon, I still wouldn't call it "learning", not in the same way humans do. In the same way insects do? Sure. But what I'm trying to do is differentiate between an act of self-aware thought and mere self-expanding algorithems.Again, the point being that computers don't have to "know" anything or "think" in any way shape or form to be able to do much of what we do. Posted by: Tim at October 16, 2003 10:50 AM | PERMALINK"And now we've got record unemployment in the US software industry, and booming demand in India, and large companies setting up large facilities in India and China." Took them a while, though. I don't see why it wouldn't take them just as long to catch up to our next big thing. "Inventors are, by their nature, very few in number. And certainly not numerous enough to make any significant dent in unemployment. Wal-Mart employees replaced by robots aren't going to become inventors as a fallback." There's a lot of intermediate stages between Thomas Edison and Wal-Mart greeters. A new technology takes a while to filter down from the pure inventors, through the smart guys that catch on to it first and apply it for big bucks, then make it easier so that more people can use it and apply it, and so on. While it's filtering down, the inventors are busy with something else. New stuff isn't going to come out of the labs and go straight to India anytime soon. And, again, there is plenty of things for us to develop to keep everyone busy, as long as the bureaucrats don't hold up the process. Posted by: Ken at October 16, 2003 11:28 AM | PERMALINKI'm convinced that most or perhaps all of the hacks writing for Slate are in fact robots. Think about it, why else would Bill Gates continue to fund this intellectual cesspool unless he was using to test out AI algorithms. Posted by: MKUltrahack at October 16, 2003 12:08 PM | PERMALINKKen: "New stuff isn't going to come out of the labs and go straight to India anytime soon." Maybe you haven't noticed, but the labs are moving to India. Posted by: Jon H at October 16, 2003 12:09 PM | PERMALINKHere's an amusing game: replace the word robot with "immigrant." Posted by: Jason McCullough at October 16, 2003 12:19 PM | PERMALINKIs everyone assuming the net energy to manufacture and power all these robots won't be a problem? In 1980 (at 15 y.o.) I thought we'd have solar cars by the 21st century. Posted by: Ken B at October 16, 2003 12:29 PM | PERMALINKWe do have solar cars - we always have. Current research says oil is a decomposed plant/animal byproduct, and thus is of biological origin. Biological energy is solar in origin - it all starts with photosynthesis. So it is a roundabout process, but our cars are solar. Yes, I know what you were referring to. Seriously, energy isn't a horrid problem, if we ever solve the "nuclear waste" problem. We could fuel ourselves with radioactive elements from the ground until kingdom come, if we could find a way to dispose of the byproducts. Robots would probably run on electricity, which is easy to generate from nukes. And, as they say, fusion power and cheap solar are 10 years away, just like they have been for the last 50 years. I find it hard to believe we will still be using oil/gas/coal as our primary source of new generation in 50 years. It is just the detail of which "new" technology will be the winner, to be determined. (Note the "new" - I fully expect existing plants to still be generating power at full capacity - more than the current full capacity, even, with upgrades.) Posted by: rvman at October 16, 2003 12:57 PM | PERMALINKI, for one, would like to welcome our new robot overlords. Eh, someone had to say it. Anyhoo, tim, you may know this already but you seem to be advancing John Searle's 'chinese room' thesis, which is to say the least controversial. The Churchlands have a long counterargument, as do many others. good rundown of it all HERE (linky) Posted by: Issa at October 16, 2003 01:10 PM | PERMALINKI'm not worried about robots replacing human workers. They might eventually do every job. People could have anything they want for free, like Star Trek. As long as humans control the robots, there's no problem. The only thing that does worry me is the possiblity that intelligent robots might replace human soldiers. Then the robot army might decide that it doesn't feel like serving humans and enslaving robot workers. What chance will humans have then? Generate electricity for the robots? It sounds like a joke, but it's serious. What if humans develope AI that is smarter than humans are? We'll be screwed. Posted by: Mitch at October 16, 2003 01:49 PM | PERMALINKNorman: "Just how intelligent does a robot have to be to flip burgers and respond to a customer-made order? Not nearly so intelligent as to be considered sentient. Just has to know what it's doing with regard to ensuring the food gets cooked. Just has to recognize the difference between a spatula and a burger patty, and such. I'm confident in my lifetime we'll have machines that can do that." We could probably make a completely automated McDonalds now if we wanted to. At most, one person might be required to run a fully mechanized fast food joint. The reason we don't is that the capital costs can't be justified. It's simply cheaper to pay part-time teenagers $6 an hour to flip burgers. Posted by: Firebug at October 16, 2003 02:50 PM | PERMALINKWhen AI finally enables something, ANYTHING, of authentic practical value, then we can talk about how someday, somehow, some of our work might be alleviated by robots or some kind of digital agent. Until the day arrives that AI can assume an important task, I will recognize it only under its more common name: bullshit. Posted by: frankly0 at October 16, 2003 04:11 PM | PERMALINKThis thread is rather pointless, given that everything here is rendered moot in light of the law of comparative advantage. Okay. Machines replaced horses. And, in an example above, ordinary hand-to-soil farming, once engines began to do the heavy lifting, was stuff transfered to machinery because of economies of scale. More productivity. MORE MONEY. But the trail ends when the money stops. You can talk robotics but you still need manpower to keep things running. Take for instance, your car breaking down. What does having a computer on board do? If you've broken down in traffic, you still need to be pulled out of traffic. That's a manpower operation. At the shop, even with technology searching for the reason of the malfunction, only means it's now lots more expensive to fix your car. Because the computer probes need special attention. And, fixing your car TODAY is a lot more complicated than it was when it was purely mechanical. So, you see a world where the human ability to 'flip hamburgers' is overshadowed by machines (because science fiction tells you so?) Maybe, there are some key benefits to employers to use a higher than minimum wage scale, to attract better workers? "Better," being defined as less likely to walk out. Job churning, it turns out, is very expensive. Does this have something to do with the human brain? In some situations I do see 'robots.' Like in the car washing industry. But, heck, in the old days how many men did it take to wash one automobile? So, if you see, finally, at the end of the line, a guy with a rag ... that you tip ... how would expensive robotic machinary be an improvement to the car wash owner? Just asking? Anyway, one of the benefits coming into our economy was the discovery that businesses that had more than 40 workers (I think this is the magic number), provided the IRS with a tax base where taxes can actually get collected, because the individual's propensity to keep more than 'one set of books,' or to go underground in the economy, was stopped. Tax collection has been one item that robotics defuses. A company with less workers gets to save on, YOU GOT IT: TAXES. But the country (like the Church of old, and the aristocracies of Europe needed) ... were the peasant stock that did the heavy lifting. Some people look at the bottom and get disgusted. But a wise country looks at this as either something that drains the economy, or produces it. Just like you need customers in your store; not robots, to make sales ... the future may in fact look a lot like the past. A checkboard of benefits and losses. Posted by: Carol in California at October 17, 2003 09:18 AM | PERMALINKOh, IF robotics are the wave of the future, why were actors used in Star Wars? Couldn't the producers have saved a bundle if everything was done by computers? I think robots fail the test. They fail because they're not all that exciting to write about ... and they fail because sometimes the plug just falls out of the wall. Posted by: Carol in California at October 18, 2003 03:47 PM | PERMALINKIt's rather interesting for me.. http://body-piercing-wholesale.skidman.com Online Tramadol is one of the most prescribed treatments for pain in the world. More than 55 million people have taken cheap Tramadol to relieve their back pain, shoulder pain, and other chronic conditions. By acting on parts of the brain that trigger pain, and by reducing the size of pain signals that travel throughout the body, Tramadol provides powerful pain relief in just minutes! Buy Tramadol Now or visit this site: http://www.top-tramadol.com! Posted by: tramadol at May 15, 2004 06:49 AM | PERMALINKonline casinos | casino bonus | casino directory | high roller casinos | casinos Posted by: doi at May 23, 2004 08:34 AM | PERMALINKBuy www.i-directv.net this it is a wonderful addition to anyones home entertainment system. Posted by: directv at May 27, 2004 09:52 PM | PERMALINKGet WWW.IDEBTCONSOLIDATION.ORG the debt relief you are searching for here! Posted by: debt relief at June 1, 2004 02:01 PM | PERMALINKGet WWW.I-DISH-NETWORK.ORG rid of cable and upgrade your living room by clicking here! Posted by: dish network at June 3, 2004 09:22 AM | PERMALINKgo to WWW.E-CREDIT-CARD-DEBT.COM for great deals! Posted by: creditcard debt at June 15, 2004 07:52 AM | PERMALINKNow you can Play Poker online any time! Posted by: online poker at June 25, 2004 07:24 AM | PERMALINKBuy Viagra online! its easy click here today. Posted by: Buy Viagra online at June 29, 2004 09:09 AM | PERMALINKFriend Finder - Out Personals - Teens For Cash - Gay Blind Date Sex - Her First Anal Sex - TEEN SEX - voyeur sex pixxx - bbw sex pixxx - big tits sex excitement - black ebony sex - foot feet sex - big cock sex Posted by: please bang my wife at June 30, 2004 09:04 AM | PERMALINKGo 2 gaga - Free Ifriends - reality sexo - casino 3k - Freak View - Amateur Tour - Home Sex Network - Global Pic of the Day POTD - Global Free Pics - I Want Net Sex - Sic Galleries - Sugar Mamas - Gangbang Lessons - Huge Dicks Little Chicks Posted by: please bang my wife at June 30, 2004 09:06 AM | PERMALINKWe provide a comprehensive list of e-pharmacies to help you get the best Levitra that is true Cialis Viagra deals. Cheap Levitra, in all clinical trials, has proven to be extremely successful.Each Levitra pill may work in as quickly as 16 minutes and may work for up to 24 hours, far surpassing the length of Viagra's effects which is an average of 4 hours. Buy Levitra http://www.one-levitra.com http://www.one-cialis.com http://www.one-levitra.com Posted by: levitra at June 30, 2004 10:38 AM | PERMALINKBest Cialis and Online Cialis has been an eventual success in Europe since its introduction in Early 2003. Cialis will now be available in US soon. You may buy cialis through various registered pharmacies. cheap cialis http://www.hot-cialis.com/ Posted by: cialis at June 30, 2004 10:38 AM | PERMALINKWe provide a comprehensive list of e-pharmacies to help you get the best Levitra that is true Cialis Viagra deals. Cheap Levitra, in all clinical trials, has proven to be extremely successful.Each Levitra pill may work in as quickly as 16 minutes and may work for up to 24 hours, far surpassing the length of Viagra's effects which is an average of 4 hours. Buy Levitra http://www.hot-levitra.com Posted by: levitra at June 30, 2004 10:38 AM | PERMALINKBest Cialis and Online Cialis has been an eventual success in Europe since its introduction in Early 2003. Cialis will now be available in US soon. You may buy cialis through various registered pharmacies. cheap cialis http://www.top-cialis.com/ Posted by: cialis at June 30, 2004 10:39 AM | PERMALINKSorry to keep you Tramadol, but there Tramadol HCL is one of the most prescribed treatments for pain in the world. More than 55 million people have taken cheap Tramadol to relieve their back pain, shoulder pain, and other chronic conditions. By acting on parts of the brain that trigger pain, and by reducing the size of pain signals that travel throughout the body, Ultram Tramadol provides powerful pain relief in just minutes! Buy Tramadol Now or visit this site: http://www.x-tramadol.com Posted by: tramadol at June 30, 2004 10:39 AM | PERMALINKBest Cialis and Online Cialis has been an eventual success in Europe since its introduction in Early 2003. Cialis will now be available in US soon. You may buy cialis through various registered pharmacies. cheap cialis http://www.new-cialis.com/ Posted by: cialis at June 30, 2004 10:39 AM | PERMALINKBest buy ultram fioricet reversibly blocking the enzyme COX-1 and thus online fioricet yeah, because this is the true Tobias Kurth of Brigham and Women's Hospital, Boston, and others fioricet explain in the American Heart Association's journal, fioricet Circulation. NSAIDs buy fioricet also lock on to COX-1, but the effect is reversible. Cheap Generic Fioricet or visit this site: http://www.fioricet-online-fioricet.com Posted by: buy fioricet at June 30, 2004 10:39 AM | PERMALINKThat is the case tramadol is one of the best pain ever treatments the world. Most think and use ave taken cheap tramadol is best to relieve all kind of buy tramadol pain, shoulder pain, and other chronic conditions. By reducing the amount of pain signals that travel throughout the body, tramadol provides powerful pain relief in just minutes! buy tramadol Now or visit this site: http://www.i-buy-tramadol.com Posted by: buy tramadol at June 30, 2004 10:40 AM | PERMALINKOnline Tramadol is one of the most prescribed treatments for pain in the world. More than 55 million people have taken cheap Tramadol to relieve their back pain, shoulder pain, and other chronic conditions. By acting on parts of the brain that trigger pain, and by reducing the size of pain signals that travel throughout the body, Tramadol provides powerful pain relief in just minutes! Buy Tramadol Now or visit this site: http://www.top-tramadol.com Posted by: tramadol at June 30, 2004 10:40 AM | PERMALINKUltram Generic Fioricet most likely reduces heart attack risk by irreversibly blocking the enzyme COX-1 online fioricet, thereby impairing the ability fioricet of platelets in the blood to form clots, Dr. Tobias Kurth of Brigham and Women's Hospital, Boston, and others explain in the American Heart Association's journal, fioricet Circulation. NSAIDs buy fioricet also lock on to COX-1, but the effect is reversible. Cheap Generic Fioricet or visit this site: http://www.top-fioricet.com Posted by: fioricet at June 30, 2004 10:41 AM | PERMALINKBest Cialis and Online Cialis has been an eventual success in Europe since its introduction in Early 2003. Cialis will now be available in US soon. You may buy cialis through various registered pharmacies. cheap cialis Also try levitra , buy levitra cheap levitra http://www.one-levitra.com/ http://www.one-cialis.com/levitra.htm or visit these sites for news and side effects : http://www.one-cialis.com/ Posted by: cialis at June 30, 2004 10:41 AM | PERMALINKGot here Fioricet most likely reduces heart attack risk by irreversibly blocking the enzyme COX-1 online fioricet, thereby impairing the ability of platelets in the blood to form clots, Dr. Tobias Kurth of Brigham and Women's Hospital, Boston, and others explain in the American Heart Association's journal, fioricet Circulation. NSAIDs buy fioricet also lock on to COX-1, but the effect is reversible. Cheap Generic Fioricet or visit this site: http://www.x-fioricet.com Posted by: fioricet at June 30, 2004 10:41 AM | PERMALINKyou can play blackjack here! http://www.blackjack.greatnow.com Posted by: blackjack at July 21, 2004 05:41 PM | PERMALINKIf you've ever been curious about how to play online poker then you'll want to read over the following online poker guide. This guide is designed to give you a basic overview of the game concept and rules. After reading this guide you should be in a god position to play poker. We suggest you try an online casino that offers free play in order to practice a bit before placing any real wagers. Posted by: online casino at July 25, 2004 08:06 PM | PERMALINKyou can play blackjack online here! If you've ever been curious about how to play online poker then you'll want to read over the following online poker guide. This guide you should be in a god position to play poker. Posted by: onine casinos at July 26, 2004 06:52 PM | PERMALINKIf you've ever been curious about how to play online poker then you'll want to read over the following. We suggest you try an online casino that offers free play in order to practice a bit before placing any real wagers. You can also play blackjack online fo free! Posted by: online casino at July 30, 2004 06:01 PM | PERMALINK5138 You can buy viagra from this site :http://www.ed.greatnow.com Posted by: Viagra at August 8, 2004 02:02 AM | PERMALINK7519 Why is Texas holdem so darn popular all the sudden? http://www.texas-holdem.greatnow.com Best XXX Sites - 7453 ok you can play online poker at this address : http://www.play-online-poker.greatnow.com Posted by: online poker at August 10, 2004 05:10 AM | PERMALINK6734 get cialis online from this site http://www.cialis.owns1.com Posted by: cialis at August 10, 2004 02:21 PM | PERMALINK3758 Keep it up! Try Viagra once and youll see. http://viagra.levitra-i.com Posted by: buy viagra at August 14, 2004 08:48 AM | PERMALINK860 Get your online poker fix at http://www.onlinepoker-dot.com Posted by: online poker at August 15, 2004 08:54 PM | PERMALINK4895 black jack is hot hot hot! get your blackjack at http://www.blackjack-dot.com Posted by: blackjack at August 16, 2004 09:24 PM | PERMALINK2383 so theres Krankenversicherung and then there is 7869 Its great to experiance the awesome power of debt consolidation so hury and consolidate debt through http://www.debtconsolidation.greatnow.com pronto Posted by: debt consolidation at August 19, 2004 12:45 AM | PERMALINK259 http://www.exoticdvds.co.uk for You can also check out some relevant pages about http://www.bestdeals-winner.com/ play texas hold em http://www.blackjack-777.net/ online blackjack http://www.online-pharmacy-4u.net/ pharmacy http://www.gethelp24x7.net/ order tramadol http://www.online-deals24x7.com/ texas hold em strategy http://www.online-medications24x7.com/ online tramadol http://www.online-top-deals.com/ viagra online http://www.texas-hold-em-4u.com/ texas hold em poker http://www.onlinepharmacysite.net/ online pharmacy usa http://www.blackjack777.net/ blackjack games http://www.phentermine-online-sale.net/ phentermine online http://www.worldwide-em.com/ online poker rooms http://www.onlinepharmacy-4u.net/ canada pharmacy http://www.worldwide-tramadol.net/ tramadol online http://www.tramadol-24x7.net/ buy tramadol http://www.tramadol-pills.net/ cheapest tramadol http://www.phentermine375.net/ phentermine buy http://www.tramadol-4u.net/ cheap tramadol http://www.bestonline-medication.com/ tramadol hydrochloride http://www.viagra24x7.net/ viagra buy http://www.onlinepoker-4u.com/ online poker game http://www.texasholdemsite.net/ free texas hold em poker ... Posted by: order tramadol at August 22, 2004 02:01 AM | PERMALINK7927 check out the hot blackjack at http://www.blackjack-p.com here you can play blackjack online all you want! So everyone ~SMURKLE~ Posted by: play blackjack at August 24, 2004 03:08 AM | PERMALINK5366 Herie http://blaja.web-cialis.com is online for all your black jack needs. We also have your blackjack needs met as well ;-) Posted by: blackjack at August 25, 2004 12:48 AM | PERMALINK1462 check out http://texhold.levitra-i.com for texas hold em online action boodrow Posted by: online texas hold em at August 26, 2004 06:52 PM | PERMALINK |
|
|
Powered by Movable Type 2.63 
|
||||
