TECHNOS

July 27, 2008

TECHNOS QUARTERLY Spring 1998 Vol. 7 No. 1

The Leonardo Loop: Science Returns to Art

By James Bailey

During the early 1990s the author worked in the field of parallel supercomputing and wrote about the revolution it is brewing in mathematics. Now he is spending his afternoons in high school art classes. Why? Because art classrooms are where the future lies for the high achievers of science and industry in the next century.

Five hundred years ago the printing press replaced pen and ink as the communications technology of Europe. Within decades, millions of printed volumes flooded the Western world. These printed books not only changed how people communicated but also reshaped what they communicated, hence what they thought about and what they taught. For example, the printing press drove a five-hundred-year wedge between science and art, pushing the latter to the brink of extinction in the curriculum. In the book era, high achievement students were better off learning to manipulate printable symbols than they were learning to see. That seems a harsh thing to say, but it is true.

Five years ago the pixel replaced the type slug as the world’s communication technology of choice. The World Wide Web joined the Postscript® laser printer to restore the old economic balance of image and word that the medieval scribe had taken for granted. As a result of these two innovations, pixels (the little individual dots of color on your computer screen) are no longer relegated to just forming up letters and digits on behalf of a word processor or spreadsheet as they were a decade ago. Pixels, and the bits behind them, have a lot more to offer than that. For example, they are supporting the rise of a whole new wave of parallel intermaths, such as neural networks and genetic algorithms, that will ultimately replace the old Industrial Age maths of algebra and calculus. These new intermaths, about which I have written in detail in my book After Thought, cannot even be expressed in the old letters and digits of the printing press. They presuppose the new bits and pixels.

The bits and pixels of electronic circuits are opposite to the letters and digits of books in two profound ways. First, they are parallel. They do not form up into sequences (although human engineers and programmers can constrain them to do so); they do not take turns; they do not live life one step at a time. As René Descartes once stressed, the human mind can keep only one piece of information in mind at a time and therefore is most efficient operating step by step. Old Descartes embraced the teaching of algebra precisely because it shaped young minds to process information sequentially. Bits and pixels, like life itself, routinely keep millions of things going at once.

In addition to being parallel, bits are adaptive in the same way that genes and neurons are. If you have enough of them, they can change and adapt and learn on their own. The new intermaths do this routinely, whereas the old numbers and equations of the book never could. The printed pages on which Johannes Kepler stored Tycho Brahe’s data have had that data for more than three million hours now and still have not improved it one iota.

It is not too soon for high school students to be factoring these new realities into their course selections. To be sure, the first 50 years of electronic computing have been spent recapitulating the old learning of the book era. This transitory stutterstep has allowed the old automatic course choices such as algebra, calculus, and physics to seem as valid as ever. The next 50 years, however, will be very different. Students need to look out to the edges of science to see where bits and pixels are really taking us. Equally important, students must look back and see this revolution in its historical context. Let us briefly do some of both.

The partnership of mind and pen has an emphasis on spatial relationships and on the art of seeing. Shapes are central.

Humans and their communications technology form an intellectual team and always have. For thousands of years it was a partnership of human mind and pen. During the Renaissance it changed abruptly to a partnership of mind and printed book. Now it is changing again, to a partnership of mind and electronic circuit. Each of these three teams has different innate capabilities; each gravitates to different fields of inquiry, because each processes information in different ways. For example, the circles and lines of pen-and-ink mathematics disappeared from serious science during the Renaissance, replaced by numbers and equations that were far better adapted to the printing press.

What one finds by analyzing these three teams is that, mirabile dictu, the emerging era of mind and computer conspires deeply with the early era of mind and pen and surprisingly little with the intervening period of mind and book. In the future, as in the past, it will be the ability to see, not the ability to manipulate symbols, that will matter most. This realization is what brought me from the supercomputer lab to the art classroom.

Back to Leonardo
Any attempt to understand intellectual life in the era of pen-and-ink communication technology brings us perforce to Leonardo da Vinci (1452–1519). He was the last great Western mind before the onslaught of the book. We know a lot about his thought, and we know a lot about how that thought was maligned and ignored during the reign of the printing press. Both hold valuable lessons for the student of today.

The partnership of mind and book has an emphasis on linearity and the art of sequentializing. Symbols are central.

It is not just that art and science are interchangeable in Leonardo; so are word and image. He wrote about art; he drew science; and he often did both on the same page of his notebooks. He was at the forefront of two technical innovations within imagery itself–one in the realm of dimensionality, the other in color. He brought the techniques of perspective, invented just a few decades earlier, to full fruition. His Last Supper, for example, forms a perceptual extension of the refectorium in which it was painted. For the first time, images such as these imposed a fixed viewpoint, a fixed relationship between object and viewer. Leonardo also led the Italian adoption of oil paint technology invented in the north. He saw these new paints as an additional tool for modeling three dimensions: “Painters who wish to represent the relief of things they paint must cover the surface with a half-tint, then paint in the darkest shadows and lastly the main lights.”

Those who believe that the quattrocento revolutions in viewpoint and dimensionality had no enabling impact on the later science of Kepler and Newton can safely disregard the corresponding revolutions in color and dimensionality happening today. Those who believe that painting the volume of space, known as the ether to scientists, between viewpoint and object helped scientists to imagine it as filled with fields of gravitational force will be attentive to the breakdown of the fixed viewpoint that the techniques of virtual reality are now bringing about. The advances in artistic dimensionality that are happening today are every bit as significant as those of the Renaissance. Together with advances in color science stimulated by the ubiquity of the color monitor, these new perspectival technologies are determining how we will experience vast amounts of future information, including the molecular biology of life itself.

The core of Leonardo’s understanding of the world was his focus on simultaneity, or, as we would say today, parallelism. He knew that the important elements of reality happen and must be understood all at once, not one at a time as Descartes later preached. To express simultaneity, science required art. “The poet, in describing the beauty or ugliness of a body, will describe it to you part by part and at different times, but the painter will make you see it all at the same time. The poet cannot give you in words the true shape of the parts of which the whole is composed, as can the painter, who places it before you with the same truthfulness that is in nature.”

Leonardo’s determination to express reality as it actually is spills out all over the pages of his famous manuscript notebooks. Here text and image trip over each other in a vast mosh pit of erudition. There is none of the Galilean passion for the underlying abstraction; Leonardo presents the world as it presents itself.

In so doing, of course, he disqualified himself from the printing presses that fueled Galileo’s fame. He did so consciously, recognizing that painting “does not have an infinite progeny as does the printing of books.” It is not just that he chose not to publish his science. He could not publish his science because it was literally unprintable. As Zubov has noted, it was “manuscript in its very substance…one cannot imagine it being printed by such as the house of Aldo Manuzio…. They would have returned it to Leonardo as unsuitable for printing.”

Manuscript in its very substance: To be a published scientist in the era of print, Leonardo would have had to become a different scientist, treating different subjects and expressing them in very different ways. Perhaps a bit more astronomy, certainly a lot less biology, definitely a narrative thread rather than isolated instances. Get with the program, Leonardo.

The partnership of mind and circuit has an emphasis on adaptation and the art of recognizing. Patterns are central.

To evaluate Leonardo’s achievement in a more balanced way, imagine for a moment that Gutenburg had invented the World Wide Web instead of movable type. Would there still be the condescending finale to the Leonardo exhibit now touring America, asserting that he was not a true scientist because, unlike Galileo, he did not publish? Of course not. Leonardo’s work would have been downloaded onto every schoolchild’s desktop overnight. Galileo’s lengthy dialogues, in which Salviati, Sagredo, and Simplicio prattle on endlessly, might have been lucky to get percussio per diem una. We know this because we can already see the first fruits of Leonardo’s science being reborn in our new age of bits and pixels.

Anatomy offers a fascinating example. Medical school students in the book era were traditionally exposed to more than 10,000 terms in the course of their studies. The only way to communicate with another doctor was by using these 10,000 names. Now there is ADAM®, an extraordinarily detailed virtual cadaver on CD-ROM that is becoming a standard medical school “text.” Doctors linked by the Internet can communicate by simply pointing to the virtual reality without having to name it. (The names are all there if needed: click, and they appear; but there is no longer the pressure to memorize them in advance.) After 500 years of words replacing images, images are once again cheap enough to take back their proper role from words.

Architecture and engineering are two more of Leonardo’s passions that are being revolutionized by the pixel’s ability to express the third dimension and to vary the viewpoint. Blueprinted plans and elevations are designed to guide builders; they do not give meaningful information about the visual impact a structure will have when it is actually built. At a time when the built environment is, for most of us, the whole environment, we need a more farsighted process. Modern computer-aided architects not only can foresee the visual impact of their designs; they also have the ability to design buildings they never could have contemplated before. The new Guggenheim Museum in Bilbao is one example.

The clearest example, however, of the new role of the pixel is in molecular biology and drug design. Already biology has replaced physics as the master science, just as physics replaced astronomy at the beginning of the print era. The new century will be defined by the genomes and docking sites of the biologist, not the “=” sign type slug of the physicist.

A drug designer deals in shapes. A drug is a molecule whose shape allows it to fit together with another molecule in ways that influence the behavior of living cells. Often it does so by fitting into a site where another harmful molecule likes to go. Such drug molecules act as blockers. To find such substitutes, scientists project the molecules onto computer screens, rotating them in three dimensions and looking for a fit.

The new molecular biology is also dependent on parallelism. For every molecule that might act as a blocker, there are millions whose shapes are clearly wrong, although they may be just right for something else. There is no hope of inspecting the possibilities one at a time. The whole vast range must be seen in parallel, processed in parallel, and winnowed down in parallel.

Art for Productivity’s Sake
I went to the art classroom in order to see more clearly. I was interested in art strictly for high achievement’s sake. Others have focused on the role of art in balancing out the lives of 60-hour-week overachievers. I was interested in the potential of art to make those 60 hours even more fiercely productive.

The high achievers of the next century’s science and business need to know instinctively that the world is parallel. They also need to know how to process complex visual information, because that is how information is shared between human and computer. Our eyes are the only senses we have that can keep up with electronic circuitry. One cannot learn parallelism and visual skill in math class; the old industrial age maths of algebra and calculus teach just the opposite. They teach us to fallaciously see the world as sequential and to process it in symbols, one small step at a time.

Can young minds get better preparation elsewhere? The answer is yes; resoundingly yes. What goes on in a serious high school art classroom has surprisingly little to do with aesthetics per se. It is all about seeing. It is about getting rid of the analytical mind’s opinion of what an object like a face “should” look like and boring in on what it actually does look like. “That may be pretty, but it is not what the object actually looks like. Scrape it off and do it again.” “Don’t focus just on the object. Focus on the shadows and shadings that communicate its volume.” “Don’t make it up. You are never allowed to make it up.” “Look!”

Perhaps more important, students are drilled again and again in seeing the world whole and all at once. “You have to draw the whole thing and the individual elements at the same time.” “Don’t spend so much time on that one small area because when you draw the areas around it, it will be wrong.” For three hours, three times a week, these students confront their errors in seeing and learn to process visual information more skillfully.

Sacrifice, Talent, and Cost
There will be three objections to this particular classroom example. The first is that the average public high school is completely different from the Walnut Hill School, a private high school for the arts where I have done my initial observing. I do not agree. While Jim and Holly and Ken and Linda and Michelle are extraordinary teachers, public schools have art faculty who trained to the same rigorous standards. What is perhaps different is the uniformly high motivation of their students. All too many public school students treat art class like gym class. The students I observed were more like championship athletes; they want to win and are willing to sacrifice to learn how.

The second objection will center on creative talent. The art students at Walnut Hill all have it, although their teachers rarely acknowledge this fact explicitly, because the focus is all on seeing. Students who aspire to leadership positions in science and business may not have this same level of artistic confidence, but this is where the personal computer makes all the difference. We may not all be able to capture the shadings on a sphere with a paint brush, but we can all do it with a rendering program, and we can experience multiple viewpoints as a bonus. We may not all be able to paint a flower, but we can all design a building and struggle to get each of the parts to work together harmoniously. Because the culture of the pen and the culture of the pixel conspire so deeply, future scientists and business people can join future artists in the serious art classroom, to the benefit of both. In so doing, “nonartistic” students with courage and a color laptop can vastly improve their visual skills and thereby give themselves a big competitive advantage in the high-performance fields of the next century.

All of which brings us directly to the third objection, which is cost. Education based on serious scientific visualization is viewed as elitist because it uses equipment that is still expensive. We need to turn that around: all it costs is money and less of that every year. Decades of costly public attempts to help disadvantaged children keep up in a book-based curriculum have shown that money alone cannot do it. This result is not surprising. In the book-based curriculum, you spend 10 to 20 years learning to manipulate symbols in totally artificial contexts (“If a hen and a half lays an egg and a half in a day and a half…”) before getting to anything real. The students who win are the ones who grow up in symbol-successful families that supply the conviction that it is worthwhile doing patently irrelevant assignments each night because there is a big reward at the end. At least there used to be.

Visual learning requires no such contrived novitiate. We do not start children out on toy paintings and save the real ones for graduate school. Fourth graders can, and probably should, start interacting with the same human anatomy disk that they will use later if they decide to go to medical school. A high school student can, and should, design a house as an art class project and learn to coax two molecules together on the computer screen to see if they dock. The biology class as a whole should pool the computer resources and fold a protein, even if it uses a whole year of computing time. What it takes is gumption, which is free, and the right computing equipment, which is getting cheaper every year. Then it is a level playing field where growing up in a symbol-manipulating family does not confer the ultimate advantage. Authors in the book era marveled all the more at Leonardo’s achievements because his illegitimate birth had denied him access to a university. More likely, Leonardo accomplished what he did precisely because his disadvantaged family origins shielded him from the university.

The Thing of Shapes to Come
Midway through his famous but prolix Astronomia Nova, Johannes Kepler begs the opportunity, “after so much heavy lifting,” to engage in a brief flight of speculation. Perhaps we too have earned the same right at this point.

Imagine thousands of miraculous pages of notes written by an ancient, almost mythical master. They lie untouched in a follower’s attic for decades after his death, while the power of the press takes hold and the whole focus of the educated Western world shifts. From that attic they are scattered to other attics and other collections, many to be lost forever, some to remain undiscovered until the 1960s, while a new and virulent Scholasticism takes hold. Braced by the godlike achievements of Isaac Newton, the new breed of Schoolmen assert that the truly important issues of study are those that revolve around inanimate objects. “All science is either physics or stamp collecting,” they say. The only valid form of scientific discourse is the equation: a scientific theory is only considered to be fully formed when it has been expressed as a partial differential equation, they say. The ancient master’s pages were filled with drawings, but by the time of the French Revolution, Lagrange is bragging, yes bragging, that his treatise on mechanics contains no visual images at all. The ancient master’s notebooks are filled with the sciences of life, but that is mere stamp collecting. For biology and the understanding of life, it is a darkened age indeed.

By the mid-20th century, the dogmatism of the equational Schoolmen (and yes, they are still pretty much all men at this point) spreads to the social sciences and seeks to drive out words as well as images. Whole generations of economists are raised on the Samuelson Fallacy, that economies and other living adaptive systems can be usefully characterized only by numbers and equations; words need not apply. “The laborious literary working over of essential simple mathematical concepts…involves as well mental gymnastics of a particularly depraved type,” the equational Schoolman says.

And yet. And yet. How many times the equational Schoolmen reached new heights of understanding, only to find the pen scratchings of an illegitimate old man already there! His answers, of course, came in the visual form that was unacceptable to the New Scholasticism. They were an intellectual telescope through which many modern masters refused even to look. But what answers they were! Encapsulated in lowly drawings instead of the sanctified differential equations, they cast an annoyingly sublime shadow over the whole development of Western science. Indeed, the sheer beauty of these answers was all that secured their survival. Spurned by scientists, the ancient pages were saved by wealthy dilettantes instead.

To see how dark and constricted our bookbound approach to teaching and scholarship may have become in these centuries since the Renaissance, let us look at the work of a man who would undoubtedly blush at any comparison to Leonardo. Kenneth Snelson is an acclaimed sculptor who, in the 1980s, found himself unable to make progress on two important fronts. First, the physical limitations of his sculptural materials were preventing him from shaping the realities he wanted to shape. Second, he wanted to extend his earlier work on the description of the atom, an entity which he felt had not been convincingly characterized by the incumbent methods of science. Despite the daunting cost in those days, Snelson acquired an SGI computer and has used it ever since, both to make sculptures out of pixels and to develop his atomic theory.

When I first heard Kenneth talk at a conference at Princeton, conferences were the only way to learn about his work. Other than a few scarce copies of a museum catalog and occasional video displays in science museums, his work was not, could not be, published. Now it is, at www.inetarena.com/~pdx4d/snelson/Portrait.html. A selection of his sculpture and a description of the Snelson atom are readily available for the first time. As virtual reality modeling language (VRML 2.0) becomes ubiquitous, we will be able to experience such sculptures from multiple viewpoints as well.

Kenneth Snelson on the Snelson atom: “Because it is my work to imagine and build sculptures from physical forces, the electronic atom’s form and workings have seemed a kind of sculptural riddle, and, as I see it, one not yet solved convincingly by science.”

Snelson represents a model of intellection for the new century, rooted firmly and unselfconsciously in shapes and patterns instead of symbols, insistent on the unity of art and science rather than their dichotomy, and embracing the partnership of self and computer as the key to doing wholly new things instead of just speeding up old things. For sure, philosophers will say this is just the old “empiricism versus whatever” debate, and maybe it is. But philosophers are among the most bookbound of our Schoolmen. Perhaps concepts like “versus” are helpless to characterize what is really going on. Perhaps science is not merely returning to art. Perhaps science and art together are going to wholly new places.

None of our children can fully foresee the future they will encounter. What they can know with some surety is that regardless of their career directions, they need to spend serious amounts of their time practicing and learning how to see. They also need to avoid classes that teach them to, in John Locke’s words, think in train. Five periods of high school art a week, along with five periods of biology, are essential preparation for the world of the 21st century. If that leaves time in the schedule for only two periods a week of algebra appreciation, so be it. That seems a harsh thing to say, but it is true.

“Because these computers will be free to evolve in whatever way works for them, the odds are high that they will evolve a process of intelligence that is not the same as ours, or even understandable by ours.”
—From After Thought: The Computer Challenge to Human Intelligence by James Bailey, published by BasicBooks.

ACKNOWLEDGMENTS

Leonardo images from 19th-century editions of his notebooks, courtesy of the Boston Athenaeum. Images of people ©Photo Disc. Modern anatomy images ©ADAM Software Inc. Modern vehicle image ©Abvent. Snelson atom ©Kenneth Snelson. Author photo ©John Chomitz. All other images, text, and image compositions ©James Bailey.

James Bailey (bailey@tiac.net) is an independent scholar focusing on the role of electronic computing in the overall history of ideas. He was formerly a senior manager of Thinking Machines Corporation, manufacturer of the 64,000 processor Connection Machine parallel computer. His book After Thought chronicles the coming impact of parallel computers on mathematics.