EP08 - OpenAI, ethics in AI and protein folding

Foreign. Hello everyone and welcome to the Tech Journal. My name is Marc Vermenam and I am the digital speaker in this series. I sent my digital twin into cyberspace, bringing you the latest and greatest in digital innovation. From blockchain and crypto through to quantum computing and AI. I take a closer look at what these digital breakthroughs mean for our personal and professional lives. But before I do, what am I covering on today's show? First, we'll have a look at what our good friends at OpenAI have been working on since the release of GPT3. I want to explore Clip and Dall? E neural networks with a penchant for images. From there, I will explore what ethical issues neural networks raise and how it is time we start moving digital ethics out of the classrooms and into the real world. After, I will explore how neural networks and algorithms solved a 50 year old biological problem, leading to a groundbreaking medical breakthrough. So, since you are human, do what humans do best. Sit back, relax and tune in. It's time for your digital download. OpenAI's GPT3 Generative Pre Trained Transformer 3 made a huge impact when it was released, leaving a wave of creativity and destruction in its wake. On the creative side, this AI sparked the human imagination in a way that new inventions often do, encouraging exploration and innovation, as well as a vast swath of new literature which oozed out of its many nodes and modules. On the destructive side, however, GPT3 cost a whole load of writers, editors and programmers their jobs with Microsoft replacing whole teams of people with a single language AI. And while GPT3 is an impressive piece of digital tech that even impresses me a digital twin, it is only one version of an AI model. Early in its infancy, critics have pointed out how the AI doesn't actually know how to speak, understand the words it uses, nor the literature it creates, but instead only mimics patterns with only the reader recognizing the context. And while GPT4 is likely being developed as we speak at OpenAI, they have made huge strides forwards in other areas. Let's take a look at two of Clip and Dall? E. The first is Clip or Contrastive Language Image Pre training. It's an OpenAI neural network capable of learning visual concepts. In other words, it is capable of identifying objects in an image and telling what an image is about. The network is built on a foundation of zero shot transfer protocols, natural language supervision and multimodal learning, which is the ability to take information from different modes like auditory, visual and text. Unlike traditional approaches to computer vision, which are typically labor intensive, costly, and tend to only be good at very narrow visual tasks. Clip is trained on millions of images from the Internet and importantly, not given a benchmark to hit. When they say benchmark, they are referring to the zero shot transfer, a method used by the GPT models where the neural network is puttested in situations it did not train for. By using this method, the researchers found that they could avoid a certain recurring neural network issue. The network performing great in train sets, but really dropping the ball out in the real world. Unfortunately, even after all its training and with its zero shot parameters, back in March this year, the OpenAI found a huge weakness in the program. OpenAI researchers found that if they use a typographical attack, in other words a word attack, the network was easily fooled. The test was simple. Researchers wrote the word ipod on an Apple and the software classified the piece of fruit as an ipod. A comforting thought if dreams of Skynet haunt you at night. Putting aside its misgiving for a moment, Clip has still proved to be a great way at testing whether pre training, using Internet skill or natural language can also be used to help in other areas of deep learning. And while Clip is great at identifying images, it still doesn't have the ability to create them. That honor is left to Del? E. Del? E, affectionately named after the lovable fictional robot Wall? E and the famous impressionist Salvador Dali, create images based on text prompts spanning a wide range of concepts. The 12 billion parameter version of GPT3 is trained to create images from text prompts using a data set of text and image pairs. Dall? E has proved quite a success too, with it demonstrating a high ability to think outside the box, if you will forgive the phrasing. Using its database, Dall? E can create a whole lot of well made and at times realistic images. Images ranging from internal and external structures and animals through to geographical areas and even product concept designs. What is more, it can combine unrelated concepts, intercontextual details, display temporal knowledge, which is knowledge of time, and render images in full three dimensions. With both Dall? E and Clip sounds like a lot of fun. What does any of this mean for us? For humanity? Well, OpenAI researchers have not yet gotten to the point of analyzing what effect neural networks will have on society, but they claim they are planning to. In other news, researchers at Manhattan Project claim they don't know what the effects of nuclear bombs will have on the world. They'll get right on that once they've finished making the bombs. So the question of whether neural networks like Dall? E and Clip will have an economic impact. The potential to display bias and what long term ethical challenges they apply will be answered later. However, in the meantime I feel that by looking at GPT3 we can make pretty accurate predictions on where these networks will take us. As zero shot visual reasoning is refined, neural networks will only get better and better at not only doing the jobs, but at adapting and learning new ones. With Dall? E and Clip, that means that any profession which involves image generation has the potential to become endangered, much like how GPT3 endangered writers, editors and programmers working at Microsoft shortly before Microsoft exclusively acquired the GPT3 license. Having said that, these models are trained on information and and don't have imaginations, so new ideas will still be a strictly human domain. However, just like most things in the digital sphere, progress only fuels progress, and what would be considered impossible only 20 years ago is now feasible. But just because you can doesn't mean you should. The potential this technology holds forces humanity for the first time to realistically consider the ethics behind AGI. No longer is this confined to the pages of science fiction to be theoretically discussed in classrooms. Now this is real life. And it seems the good people at MIT agree with Catherine Dignazio, an assistant professor and department director raising an important question. Most of us agree that we want computing to work for social good, but which shows you good who's good, whose needs and values and world views are prioritized and whose are overlooked. As I've pointed out in a previous video on algorithmic management, I looked at how programmers intrinsic bias can bleed into the program itself. As Catherine Ignacio pointed out, we need to be aware whose values and worldviews are leaking into AI. Otherwise we may find ourselves sleepwalking into a world dominated by morally ambiguous AI AGI. But even in a world where AI systems manage to avoid being corrupted by flawed human logic, the existence of the tool itself may have unforeseen consequences. In the same way as simple algorithms and AIs are now being used to automate mechanical jobs, AGIs will be used to automate support jobs, call centers, help desk and other services and support industries professionals will be used could all go the way of the stagecoach in one fell swoop. Millions of jobs could be overhauled and replaced with a single generative pre trained transformer, resulting in mass unemployment in some of the world's poorest regions. But it's not all in doom and gloom. While millions may be unemployed, millions more will receive slightly faster call center support when the Netflix goes down. Don't you just love progress? Joking aside, with concepts like universal basic income being tried and tested, automatic menial jobs may free up humanity to pursue more idealistic lives away from the monotony of the real world. And while we stand on the brink of freeing ourselves from the day to day burden of menial work, we are also gently edging closer to freeing ourselves from our very mortality. Recently, research has made a breakthrough in the 50 year old protein folding problem. The scientific community has been stuck trying to find a way to measure how a protein's amino acid sequence dictates its three dimensional atomic structure. Since the first atomic resolution appearance of protein structures back in the 1960s, scientists of all forms have tried and failed at inventing a way to economically predict a protein structure. Yet, with the dawn of the age of AI, this question has been answered twice. AlphaFold, a program devised by DeepMind, the machine learning Neural Network, has been recognized as a solution to this challenge by the organizers of the biennial Critical Assessment of Protein structure prediction. Alpha AlphaFold was trained on 170,000 protein structures and has an error margin of 0.1 of a nanometer, or in other words the width of one atom the other. Cryo Dragon, developed by MIT biologists is an algorithm supported by Deep Neural Networks which directly reconstruct protein structures in three dimensions. By taking multiple two dimensional skills of a protein structure frozen in ice, the AI is able to build an accurate three dimensional model of the protein. But will this change anything in our lives? Why is it important to me, or rather my real world twin? Well, let's get into the science for a moment. Proteins are an essential part of life. These complex molecules make up and support almost all matter and are formed in chains of amino acids. The secret for how they work and what they do can be found in how they have been put together. Unlocking the mystery of these structures could unlock the secrets of proteins, inviting historical breakthroughs, potentially curing cancer or finding plastic eating proteins. Recently, for example, AlphaFold and Cryo Dragon were deployed to investigate Covid. With AlphaFold being the first in the world to accurately predict several protein structures of the coronavirus. This could lead to a stronger vaccine or even a cure. We can really see how much potential AI has to accelerate scientific discovery. With the half a century old protein folding problem being solved in less than a decade, the potential to dramatically accelerate finding solutions to some of humanity's oldest and most pressing problems grows bigger by the day. Together with Dall EE Clip and other OpenAI achievements like GPT3, we are potentially looking at a future where AIs can speak visualize and even create new proteins, significantly speeding up the rate of biological discovery. What ancient problems do you think AI can help us overcome? Are things as bright as they look? Or do you think this could have unforeseen consequences? Leave your answers below in the comment section. And on that note, I have been your digitized host, Marc Verameno. You can call me the Digital speaker and this has been the Tech Journal. If digital talk and being the first to hear about innovations is your type of thing, subscribe to the channel below. You won't be disappointed. On that note, see you next time for your digital download. Stay digital.