BICYCLE:MIND::engine:imagination
August 5, 2022 1:49 AM Subscribe
An interview with Midjourney founder David Holz - "AI-powered image generators like OpenAI's DALL-E and Google's Imagen are just beginning to move into the mainstream. [Holz] explains how the technology works and how it's going to change the world."
also btw...
Finally, an answer to the question: AI — what is it good for? - "Got a protein? This AI will tell you what it looks like."
Google's DeepMind AI Predicts 3D Structure of Nearly Every Protein Known to Science - "At last, the decades-old protein folding problem may finally be put to rest."
A.I. Predicts the Shape of Nearly Every Protein Known to Science [ungated] - "DeepMind has expanded its database of microscopic biological mechanisms, hoping to accelerate research into all living things."
Cars are faster than humans, but that doesn’t mean we stopped walking. When we’re moving huge amounts of stuff over huge distances, we need engines, whether that’s airplanes or boats or cars. And we see this technology as an engine for the imagination...David Holz, founder of AI art generator Midjourney, on the future of imaging - "Optimizing for beauty while trying to suppress sensationalism."
Within the next year or two, you’ll be able to make content in real time: 30 frames a second, high resolution. It’ll be expensive, but it’ll be possible. Then, in 10 years, you’ll be able to buy an Xbox with a giant AI processor, and all the games are dreams.[1]
[...]
Midjourney is a new source of water?
[Laughing] Yeah, that’s a little scary when you say it that way.
I think we, collectively as a species, have discovered a new source of water, and what Midjourney is trying to figure out is, okay, how do we use this for people? How do we teach people to swim? How do we make boats? How do we dam it up? How do we go from people who are scared of drowning to kids in the future who are surfing the wave?[2]
also btw...
Finally, an answer to the question: AI — what is it good for? - "Got a protein? This AI will tell you what it looks like."
Google's DeepMind AI Predicts 3D Structure of Nearly Every Protein Known to Science - "At last, the decades-old protein folding problem may finally be put to rest."
It wasn't until 1957 when scientists earned special access to the molecular third dimension.Nearly all protein structures known to science predicted by AlphaFold AI - "Let's hold off the champagne until an actual drug is developed using this tech."
After 22 years of grueling experimentation, John Kendrew of Cambridge University finally uncovered the 3D structure of a protein. It was a twisted blueprint of myoglobin, the stringy chain of 154 amino acids that helps infuse our muscles with oxygen. As revolutionary as this discovery was, Kendrew didn't quite open up the protein architecture floodgates. During the next decade, fewer than a dozen more would be identified.
Fast-forward to today, 65 years since that Nobel Prize-winning breakthrough.
On Thursday, Google's sister company, DeepMind, announced it has successfully used artificial intelligence to predict the 3D structures of nearly every catalogued protein known to science. That's over 200 million proteins found in plants, bacteria, animals, humans — almost anything you can imagine.
A.I. Predicts the Shape of Nearly Every Protein Known to Science [ungated] - "DeepMind has expanded its database of microscopic biological mechanisms, hoping to accelerate research into all living things."
The technology is not perfect. But it can predict the shape of a protein with an accuracy that rivals physical experiments about 63 percent of the time, according to independent benchmark tests. With a prediction in hand, scientists can verify its accuracy relatively quickly.Google's DeepMind AI cracks 3D structure of nearly all proteins known to science in major breakthrough - "For example, protein structures predicted by AlphaFold are helping in the development of drugs for neglected tropical diseases like leishmaniasis and Chagas disease – illnesses that disproportionately affect people in poorer parts of the world. And in April, scientists at the Yale University used AlphaFold's database to develop a new Malaria vaccine."
Kliment Verba, a researcher at the University of California, San Francisco, who uses the technology to understand the coronavirus and to prepare for similar pandemics, said the technology had “supercharged” this work, often saving months of experimentation time. Others have used the tool as they struggle to fight gastroenteritis, malaria and Parkinson’s disease.
The technology has also accelerated research beyond the human body, including an effort to improve the health of honeybees. DeepMind’s expanded database can help an even larger community of scientists reap similar benefits.
Decoding protein structures do not just aid in curing diseases but can also help engineer solutions for global environmental issues. For instance, researchers have joined hands with DeepMind’s AI to develop faster-acting enzymes to break down and recycle some of the world’s most polluting single-use plastics.[3]'The entire protein universe': AI predicts shape of nearly every known protein - "DeepMind's AlphaFold tool has determined the structures of around 200 million proteins."
“AlphaFold is the singular and momentous advance in life science that demonstrates the power of AI. Determining the 3D structure of a protein used to take many months or years, it now takes seconds,” Eric Topol, Founder and Director of the Scripps Research Translational Institute, said.
“AlphaFold has already accelerated and enabled massive discoveries, including cracking the structure of the nuclear pore complex. And with this new addition of structures illuminating nearly the entire protein universe, we can expect more biological mysteries to be solved each day,” Dr Topol added.
Martin Steinegger, a computational biologist at Seoul National University who helped to develop a cloud-based version of AlphaFold, is excited about seeing the database expand. But he says that researchers are still likely to need to run the network themselves. Increasingly, people are using AlphaFold to determine how proteins interact, and such predictions are not in the database. Other predictions that are not currently there include microbial proteins identified by sequencing genetic material from soil, ocean water and other ‘metagenomic’ sources.How AlphaFold can realize AI's full potential in structural biology - "To make the most of artificial intelligence, data and software must be freely shared, and computational, theoretical and experimental researchers must work together closely."
Some sophisticated applications of the expanded AlphaFold database might also depend on downloading its entire 23-terabyte contents, which won’t be feasible for many teams, Steinegger says. Cloud-based storage could also prove costly. Steinegger has co-developed a software tool called FoldSeek that can quickly find structurally similar proteins and which should also be able to squash the AlphaFold data down considerably.
Even with almost every known protein included, the AlphaFold database will need updating as new organisms are discovered. AlphaFold’s predictions can also be improved as new structural information becomes available. Hassabis says DeepMind has committed to supporting the database for the long haul, and that he can see updates occurring annually.
As well as creating the tool itself, DeepMind has made policy decisions that have played a significant part in the transformation in structural biology. This includes its decision last July to make the code underlying AlphaFold open source, so that anyone can use the tool. Earlier this year, the company went further and lifted a restriction that hampered some commercial uses of the program.DeepMind found the structure of nearly every protein known to science - "Alphabet launched a company called Isomorphic Labs that will aim AI tools at drug discovery, and while it's separate from DeepMind, the two companies will collaborate."
It has also helped to establish, and is financially supporting, the AlphaFold database maintained with EMBL-EBI. DeepMind chief executive Demis Hassabis, his team, and their external collaborators deserve to be commended for this commitment to open science.
Last month, the company announced that it is establishing a research lab at the Francis Crick Institute, a flagship biomedical research centre in London. This is another welcome move, which will help to create and strengthen the close partnerships that are needed between researchers specializing in computational methods and those working more with hands-on tools.
AlphaFold on its own has limitations, as its designers fully acknowledge. For example, it is not designed to predict how a protein’s shape is altered by disease-causing mutations. It was also not originally intended to predict how proteins change shape when they interact with other proteins — although researchers are making progress on this next-generation challenge. And it’s not yet clear whether AlphaFold’s predictions will reliably provide the fine-grained detail necessary for drug discovery, such as the precise shape of the area on a protein to which a small molecule might bind — the kind of information that researchers in drug development crave.
Edit: It looks like there is an open source alternative DALLE2-pytorch with several pretrained models. The accompanying HackerNews story estimates based on the research paper it would be ~$300k in GPU time to train the models.
posted by geoff. at 2:24 AM on August 5, 2022 [2 favorites]
posted by geoff. at 2:24 AM on August 5, 2022 [2 favorites]
I've been burned too many times by over-hyped, invite-only groundbreaking technical revolutions.It's a little unclear from your comment whether you're aware that Midjourney is fully accessible to the public and that you can pay for unlimited access. It still has word filters however.
I've been burned before as well, but after using MidJourney for a few months and having had access to Dalle-2 I can unequivocally say that this technology *will* change our relationship to images in some fundamental way, I'm just not sure how.
One of the points in the lead article is how MidJourney has (for better or worse) a social dynamic not present in Dalle-2. Because it uses Discord (basically Slack on speed) there is a fascinating interplay between the community members.
Inside the community, you have a million people making images, and they’re all riffing off each other, and by default, everybody can see everybody else’s images.You have to pay extra to pull out the community — and usually, if you do that, it means you’re some type of commercial user. So everyone’s ripping off each other, and there’s all these new aestheticsThere is definitely a benefit for creative types in this (IMO): a web designer friend of mine has begun incorporating this into his design process using it for mood boards. As a developing painter I've been using it to break out of compositional ruts. It's still unclear to what degree this is novelty versus lasting value. But one thing is for sure, this is here to stay.
posted by jeremias at 4:56 AM on August 5, 2022 [1 favorite]
Cars are faster than humans, but that doesn’t mean we stopped walking.America’s suburban sprawl, disproportionate funding for car infrastructure, and all its racist effects on communities make this a worrying analogy. AI these days is still mostly “bias at scale” and even though the outputs can be amusing or magical they’re fundamentally massive amplifiers of a lot of unexamined stuff.
posted by migurski at 5:05 AM on August 5, 2022 [11 favorites]
Ah I stand corrected I didn't quite understand what Midjourney was about. It is interesting it has a very cyberpunk aesthetic compared to OpenAI and Google. Thanks!
posted by geoff. at 5:15 AM on August 5, 2022
posted by geoff. at 5:15 AM on August 5, 2022
I toned it down a bit, but I present to you, "bear high on cocaine in a jungle with a minigun" ... my life is complete.
posted by geoff. at 5:22 AM on August 5, 2022 [1 favorite]
posted by geoff. at 5:22 AM on August 5, 2022 [1 favorite]
Maybe OpenAI will cure all human disease. Maybe it will kill off any remaining sense of shared reality and destroy society. All I know is, I got it to make Muppet Mitt Romney.
posted by saturday_morning at 5:37 AM on August 5, 2022 [10 favorites]
posted by saturday_morning at 5:37 AM on August 5, 2022 [10 favorites]
See the next step is for it to generate entire movies. "Muppet Mitt Romney plays a cat and mouse game with gangster cocaine bear in the style of Michael Mann's Heat" ... then I never will leave my house. "Last season of Lost but without the stupid ending and somehow make it tie into oh I don't know Sopranos"
posted by geoff. at 5:40 AM on August 5, 2022 [1 favorite]
posted by geoff. at 5:40 AM on August 5, 2022 [1 favorite]
I would like to use a tool like this to fix the resolution on all my pre-10-megapixel cell phone pictures.
And, I wonder if ai could be used as a compression algorithm of sorts, up scaling content to super hd
posted by rebent at 5:46 AM on August 5, 2022
And, I wonder if ai could be used as a compression algorithm of sorts, up scaling content to super hd
posted by rebent at 5:46 AM on August 5, 2022
Missed opportunity to make Kermit Romney.
posted by a car full of lions at 5:59 AM on August 5, 2022 [1 favorite]
posted by a car full of lions at 5:59 AM on August 5, 2022 [1 favorite]
Rebent, you can use vance.ai to fix the resolution on your pictures. There are a number of similar AI tools like ESRGAN and SWINIR that do a surprisingly good job of upscaling.
posted by The Half Language Plant at 6:05 AM on August 5, 2022 [1 favorite]
posted by The Half Language Plant at 6:05 AM on August 5, 2022 [1 favorite]
I am thoroughly tired of the flood of bad art permeating social media. We’ve automated the production of crappy art and Just Two Things style mashups, and we’re all paying the price. Keep your lazy ideas to yourself, people.
posted by zamboni at 6:07 AM on August 5, 2022 [5 favorites]
posted by zamboni at 6:07 AM on August 5, 2022 [5 favorites]
In a way, using Midjourney is like casting a magic spell. You have to phrase things just right and add in all sorts of extra switches and hints to get something magnificent, and it often takes lots of iteration to guide the process. It's so complex that there's no way to predict the outcome without actually doing it. Discord is a miserable interface for this, however.
posted by jabah at 7:01 AM on August 5, 2022 [5 favorites]
posted by jabah at 7:01 AM on August 5, 2022 [5 favorites]
It is a pretty incredible mindfuck as baby illustrator/concept artist to see all the hot-takes about Midjourney/Dall-E killing off illustrator/concept artist jobs on social media. Yeah, I trained for the last 4 years to get to where I am, and, huh, it's all gone now?
That being said, I don't think the bulk of said jobs are seriously at risk in the short term - the current state of AI art is still too unresolved to be usable in most cases. Some artists use them to generate moodboards/ref pics, but for now I've found them to be too literal to the text prompts - I know that, at least for Midjourney, there's some parameters you can tweak to make the results more autonomous, but at some point I got annoyed and went back to perusing Pinterest and Artstation.
There's a bonus to knowing where my references come from - this way I know where to get more of the same stuff, know who to credit, and, occasionally, learn where *they* got their inspiration from. AI art just muddles up this chain of referencing/inspiring/stealing for me.
posted by seapig at 7:03 AM on August 5, 2022 [2 favorites]
That being said, I don't think the bulk of said jobs are seriously at risk in the short term - the current state of AI art is still too unresolved to be usable in most cases. Some artists use them to generate moodboards/ref pics, but for now I've found them to be too literal to the text prompts - I know that, at least for Midjourney, there's some parameters you can tweak to make the results more autonomous, but at some point I got annoyed and went back to perusing Pinterest and Artstation.
There's a bonus to knowing where my references come from - this way I know where to get more of the same stuff, know who to credit, and, occasionally, learn where *they* got their inspiration from. AI art just muddles up this chain of referencing/inspiring/stealing for me.
posted by seapig at 7:03 AM on August 5, 2022 [2 favorites]
DALL-E and MidJourney are incredibly empowering technologies with significant limitations. I've been playing with both for about a month now and they are very addictive. Often what happens is I'll have some idea, type a prompt, and the AI will come up with two types of results: Something that is close to what I had in mind but off in some significant way, and something that is way off from what I had in mind but super interesting. If it's the former, I tend to try to "chase" after my original vision by tweaking the prompt. If it's the latter, I tend to explore variations of the most interesting results. Either way it's down a rabbit hole. Chasing an original vision can get frustrating because the tools just aren't sophisticated enough to produce many classes of images. For instance, you can easily make a character (although matching eyes and realistic hands are notoriously difficult in MidJourney), but then trying to place that character in a specific scene is rough.
I recently made a DALL-E based project where I found a kind of sweet spot of creating a class of images (imaginary, but photorealistic, wooden sculptures) which I think shows off the huge space of possibilities these tools are capable of within a constrained subject.
As David Holz stated in his interview, these tools are getting better very quickly. I imagine conversation we have about them in just a year or two from now will be very different from today.
posted by gwint at 7:06 AM on August 5, 2022 [5 favorites]
I recently made a DALL-E based project where I found a kind of sweet spot of creating a class of images (imaginary, but photorealistic, wooden sculptures) which I think shows off the huge space of possibilities these tools are capable of within a constrained subject.
As David Holz stated in his interview, these tools are getting better very quickly. I imagine conversation we have about them in just a year or two from now will be very different from today.
posted by gwint at 7:06 AM on August 5, 2022 [5 favorites]
Those are amazing, gwint, and a good reminder that I should check out projects more often.
posted by mollweide at 7:39 AM on August 5, 2022 [2 favorites]
posted by mollweide at 7:39 AM on August 5, 2022 [2 favorites]
There was a recent front page post via the Projects page on the excellent autoexec.cat instagram account that posts Midjourney-generated cats. It's... extremely good.
posted by BuddhaInABucket at 7:54 AM on August 5, 2022
posted by BuddhaInABucket at 7:54 AM on August 5, 2022
In a way, using Midjourney is like casting a magic spell. You have to phrase things just right and add in all sorts of extra switches and hints to get something magnificent
We've been playing with app.wombo.art and yes, this is very much a thing. You don't have control, so much as influence, over the results. The more specific your goal the more difficult (potentially impossible) it is to achieve.
Last weekend I uploaded an old photo of myself and generated various characters from it using Wombo's (rather poorly named) "Realistic" style. While the majority of results were garbage, after many tries I had a handful of excellent RPG character portraits (albeit some with weird vaguely Daliesque details). But some concepts I could just not convince it to work with. "Pirate" was always some kind of skeletal horror, making me wonder if its training data consisted of a couple of specific scenes from Disney movies.
posted by Foosnark at 8:35 AM on August 5, 2022
We've been playing with app.wombo.art and yes, this is very much a thing. You don't have control, so much as influence, over the results. The more specific your goal the more difficult (potentially impossible) it is to achieve.
Last weekend I uploaded an old photo of myself and generated various characters from it using Wombo's (rather poorly named) "Realistic" style. While the majority of results were garbage, after many tries I had a handful of excellent RPG character portraits (albeit some with weird vaguely Daliesque details). But some concepts I could just not convince it to work with. "Pirate" was always some kind of skeletal horror, making me wonder if its training data consisted of a couple of specific scenes from Disney movies.
posted by Foosnark at 8:35 AM on August 5, 2022
Sometimes I wonder why these big tech companies are investing billions into finding ways to replace the work of artists, writers, and knowledge workers. Where are we headed with this?
posted by The Half Language Plant at 10:01 AM on August 5, 2022 [3 favorites]
posted by The Half Language Plant at 10:01 AM on August 5, 2022 [3 favorites]
When we’re moving huge amounts of stuff over huge distances, we need engines,
I'm reminded of this "Golden Ratio":
I'm reminded of this "Golden Ratio":
We recognize that efficiency is a function of vehicle weight as compared to its cargo weight - another form of the Golden Ratio. The principle is simple: If the cargo weighs more than the vehicle itself - as in a person riding a bicycle - then you are approaching true efficiency. The trick is to have a vehicle/cargo ratio of LESS THAN 1.posted by aniola at 10:29 AM on August 5, 2022 [2 favorites]
Person riding a bicycle: 30 lb. bike / 150 lb. person = .20 YES
Truck Trike: 250 lb. vehicle / 750 lb. cargo plus driver = .33 YES
Automobile carrying 2 passengers: 3000 lb. vehicle weight / 300 lb. passengers = 10 NO
An inverted utopia where robots are empowered to pursue art by the manual labor of humans.
posted by Pyry at 10:31 AM on August 5, 2022 [6 favorites]
posted by Pyry at 10:31 AM on August 5, 2022 [6 favorites]
Because it uses Discord (basically Slack on speed) there is a fascinating interplay between the community members.
This AI is being trained by corporate social media with a significant amount of bigoted content? If I'm reading this correctly it's strong confirmation for my "information pollution needs to be regulated like environmental pollution" hypothesis.
posted by viborg at 11:22 AM on August 5, 2022 [1 favorite]
This AI is being trained by corporate social media with a significant amount of bigoted content? If I'm reading this correctly it's strong confirmation for my "information pollution needs to be regulated like environmental pollution" hypothesis.
posted by viborg at 11:22 AM on August 5, 2022 [1 favorite]
The only reason anything ever gets automated is so that they can stop paying people to do it. And the mass produced stuff is always inferior, but it always ends up replacing what came before it anyway, because 'cheaper and close enough' beats the real thing every time.
Surely the only real basis for art is human psychological and emotional states. But okay, let's replace it with Sorcerer's Apprentice Art, made unbelievably fast by our idiot servants who don't know what they're doing. I'm confident the wide-reaching unforeseen effects of this will be uniformly positive. :P
posted by Sing Or Swim at 11:46 AM on August 5, 2022 [2 favorites]
Surely the only real basis for art is human psychological and emotional states. But okay, let's replace it with Sorcerer's Apprentice Art, made unbelievably fast by our idiot servants who don't know what they're doing. I'm confident the wide-reaching unforeseen effects of this will be uniformly positive. :P
posted by Sing Or Swim at 11:46 AM on August 5, 2022 [2 favorites]
This AI is being trained by corporate social media with a significant amount of bigoted content?
No, that's not the case. The Discord server is used both as the interface for creating images (as a user you literally type in a command like "/imagine a cat sitting on a throne" and an image will be generated and appear in the thread. In addition, there are chatrooms of users discussing various topics. It's a little weird that MidJourney is using a 3rd party tool as the interface to their product, but I guess the idea was that "community" was so important to the company that they thought it was a good idea.
Now the source data for the "model" (i.e. the thing that turns the text "a cat sitting on a throne" into an image) is another matter. Those image/description pairs came from who knows where (scraped from the internet, most likely.) That's where the potential bias comes from, not from the Discord servers.
posted by gwint at 12:18 PM on August 5, 2022 [1 favorite]
No, that's not the case. The Discord server is used both as the interface for creating images (as a user you literally type in a command like "/imagine a cat sitting on a throne" and an image will be generated and appear in the thread. In addition, there are chatrooms of users discussing various topics. It's a little weird that MidJourney is using a 3rd party tool as the interface to their product, but I guess the idea was that "community" was so important to the company that they thought it was a good idea.
Now the source data for the "model" (i.e. the thing that turns the text "a cat sitting on a throne" into an image) is another matter. Those image/description pairs came from who knows where (scraped from the internet, most likely.) That's where the potential bias comes from, not from the Discord servers.
posted by gwint at 12:18 PM on August 5, 2022 [1 favorite]
So I spent some time recently noodling around with the Midjourney AI trying to get human figures that were not horrifying nightmarescapes. An entertaining and informative Twitter thread by author and artist Ursula Vernon. Follow up thread.
posted by EvaDestruction at 12:28 PM on August 5, 2022 [3 favorites]
posted by EvaDestruction at 12:28 PM on August 5, 2022 [3 favorites]
I have been noodling around with Midjourney for about 3 weeks now and getting some fantastic stuff. None of it is, for me, a final image: instead these all come out as great bases from which to make paintings. And like the Twitter threads from Ursula Vernon posted above by EvaDestruction, it takes a lot of iterations and experimentation to come up with useful images.
If you search Artstation for "midjourney," you'll find a lot of people posting stuff direct from Midjourney, and you'll also find people working it into their workflows. (From my own observations, I see more professional artists using Midjourney as a step and more amateurs just posting the Midjourney piece.) That being said, I've also spotted at least two Midjourney images on Reddit from people not saying they were Midjourney (I've done enough to be reasonably sure when I spot one at this point especially when they're significantly different from other art the person has posted), although neither of them were trying to get art commissions or trying to sell the pieces so we'll see how things shake down.
Here's a video of artist Rachel Bradley showing her process of using Midjourney to generate compositions and ideas and how she incorporates the image produced into a final piece.
posted by telophase at 2:14 PM on August 5, 2022 [1 favorite]
If you search Artstation for "midjourney," you'll find a lot of people posting stuff direct from Midjourney, and you'll also find people working it into their workflows. (From my own observations, I see more professional artists using Midjourney as a step and more amateurs just posting the Midjourney piece.) That being said, I've also spotted at least two Midjourney images on Reddit from people not saying they were Midjourney (I've done enough to be reasonably sure when I spot one at this point especially when they're significantly different from other art the person has posted), although neither of them were trying to get art commissions or trying to sell the pieces so we'll see how things shake down.
Here's a video of artist Rachel Bradley showing her process of using Midjourney to generate compositions and ideas and how she incorporates the image produced into a final piece.
posted by telophase at 2:14 PM on August 5, 2022 [1 favorite]
Here's a good thread of a diverse set of MidJourney examples
posted by gwint at 2:22 PM on August 5, 2022 [1 favorite]
posted by gwint at 2:22 PM on August 5, 2022 [1 favorite]
I've been playing with Dall-e 2 for about a month. It's interesting how it can expose genres that you don't know exist. For instance "My dog looks like [whatever]" makes photos of dogs that look like whatever, but they are *also* photos of people's houses, couches, legs, carpets, floors, toys. It's almost never just the dog.
AI art is already good enough to replace nephew art and stock art for most applications, and it's only going to improve. I don't think it'll fully replace actual human artists, and especially not artists working in traditional media, because a lot of what people are buying there is connection to the artist, bragging rights, a real physical thing, maybe virtuosity or a particular point of view. And so long as machine art is not copyrightable, it won't replace commercial artists in situations where people want to own the copyright or have an exclusive license.
For this reason, I'm starting to believe it is essential that AI art remains uncopyrightable. To my mind that benefits working artists, and it also benefits everyone who maybe *doesn't* need full copyright protections (meme makers, or people who want a cool picture for a one off birthday card, or whatever.)
The people who might want AI art to be copyrightable are the ones making the AI, or anyone looking to be the next Getty images, only with machine art, and, basically, fuck those people.
(I might comment about it in every thread about I come across, as if I were Cato complaining about Carthage, if I can figure out how to be pithy about it.)
posted by surlyben at 2:36 PM on August 5, 2022 [1 favorite]
AI art is already good enough to replace nephew art and stock art for most applications, and it's only going to improve. I don't think it'll fully replace actual human artists, and especially not artists working in traditional media, because a lot of what people are buying there is connection to the artist, bragging rights, a real physical thing, maybe virtuosity or a particular point of view. And so long as machine art is not copyrightable, it won't replace commercial artists in situations where people want to own the copyright or have an exclusive license.
For this reason, I'm starting to believe it is essential that AI art remains uncopyrightable. To my mind that benefits working artists, and it also benefits everyone who maybe *doesn't* need full copyright protections (meme makers, or people who want a cool picture for a one off birthday card, or whatever.)
The people who might want AI art to be copyrightable are the ones making the AI, or anyone looking to be the next Getty images, only with machine art, and, basically, fuck those people.
(I might comment about it in every thread about I come across, as if I were Cato complaining about Carthage, if I can figure out how to be pithy about it.)
posted by surlyben at 2:36 PM on August 5, 2022 [1 favorite]
Examples of Midjourney experimentation issues--asking for a kitten on a lonely path is trouble because it's not entirely sure how many eyes your typical kitten has, and when I specify that the kitten has eyes, it loses track of how many ears a kitten usually has.
Today I've managed to get two fantastic pieces out of it, but it first took all day experimenting with parameters to get a combo of art styles and keywords to get an art style that did not produce eldritch horrors, and even after that, it took me forty-eight iterations to produce The Saint of Trusts, who is still not 100% useful and needs overpainting and zhuzhing to be coherent (see his missing shoulder and the weird lines on and coming out of his forehead). Title from my random fantasy novel title generator, because Midjourney works better with vague, atmospheric prompts instead of specific requests.
The second piece I got is a lovely character design/concept art piece of a black woman in a blue gown, but (1) it took me a long time to get her into a blue gown that looked like an actual dress and not like random fabric draped around, and (2) it's not actually what I asked for! I was trying to produce a picture of two people, characters from a book I'm writing. The prompt was "portrait 17th century mercenary white man with tattoos and scars. black woman scholar in blue gown. atmospheric. dramatic raking light. cinematic. by michael whelan. by alan lee. by mucha. by craig mullins. concept art. renaissance. painterly."
So if you're trying to get something specific, it's going to take more time than you expect and you probably won't get what your looking for. I highly doubt that at this stage anyone is going to set themselves up as an artist and take commissions or professional art for book covers (the field I know best) or representational things like that because people who commission that sort of art have specific requests that it's going to be nigh-impossible to get. On the other hand, I predict a flood of images produced by Midjourney have already started to go into microstock sites.
If you're into happy accidents and using this as a springboard, it's fantastic for that.
posted by telophase at 2:45 PM on August 5, 2022 [1 favorite]
Today I've managed to get two fantastic pieces out of it, but it first took all day experimenting with parameters to get a combo of art styles and keywords to get an art style that did not produce eldritch horrors, and even after that, it took me forty-eight iterations to produce The Saint of Trusts, who is still not 100% useful and needs overpainting and zhuzhing to be coherent (see his missing shoulder and the weird lines on and coming out of his forehead). Title from my random fantasy novel title generator, because Midjourney works better with vague, atmospheric prompts instead of specific requests.
The second piece I got is a lovely character design/concept art piece of a black woman in a blue gown, but (1) it took me a long time to get her into a blue gown that looked like an actual dress and not like random fabric draped around, and (2) it's not actually what I asked for! I was trying to produce a picture of two people, characters from a book I'm writing. The prompt was "portrait 17th century mercenary white man with tattoos and scars. black woman scholar in blue gown. atmospheric. dramatic raking light. cinematic. by michael whelan. by alan lee. by mucha. by craig mullins. concept art. renaissance. painterly."
So if you're trying to get something specific, it's going to take more time than you expect and you probably won't get what your looking for. I highly doubt that at this stage anyone is going to set themselves up as an artist and take commissions or professional art for book covers (the field I know best) or representational things like that because people who commission that sort of art have specific requests that it's going to be nigh-impossible to get. On the other hand, I predict a flood of images produced by Midjourney have already started to go into microstock sites.
If you're into happy accidents and using this as a springboard, it's fantastic for that.
posted by telophase at 2:45 PM on August 5, 2022 [1 favorite]
Telephase, that second one in particular is really good!
I think there are few things that are hard to wrap one's head around, I know it was for me:
In MidJourney I used the prompt "photorealistic photo portrait of Lee Scratch Perry : : 8k : : hyperrealism :: highly detailed :: octane render".
Here's that result.
Pretty amazing that it knows who Lee Scratch Perry is and gives him appropriate garb IMO, but one of the weaknesses of MJ specifically (and computer generated images generally) is the rendering of eyes. It's one of the hallmarks of the uncanny valley after all.
Well, Photoshop has a series of "Neural Filters" which use AI for various purposes, including one called "Smart Portrait". A feature of this is the ability to alter expressions or change the direction of eye gaze. (I know, I know.)
Running the first photo through this filter and adjusting eye direction ever so slightly creates a more realistic gaze. In addition, by using *another* AI tool (Topaz AI), the image can be upscaled to higher res.
Here's that result.
The main point here is that 3 different AI tools were involved in that last image, and soon some versions of all 3 will all be consolidated into one engine.
posted by jeremias at 3:43 PM on August 5, 2022 [2 favorites]
I think there are few things that are hard to wrap one's head around, I know it was for me:
- The speed at which the technology is advancing is incomprehensible, especially with the addition of community "crowdsourcing" or whatever. It's like nothing I've ever seen, and a bit frightening tbh. A year ago, AI images had that weird rainbow color HDR look which was laughable. That default look is far in the rear mirror, and the speed of the vehicle is getting faster every day.
- There's a patchwork of different AI tech out there and each have their strengths and weaknesses, but at some point in the near future they will begin to converge, leading to more sophisticated images out of the box.
In MidJourney I used the prompt "photorealistic photo portrait of Lee Scratch Perry : : 8k : : hyperrealism :: highly detailed :: octane render".
Here's that result.
Pretty amazing that it knows who Lee Scratch Perry is and gives him appropriate garb IMO, but one of the weaknesses of MJ specifically (and computer generated images generally) is the rendering of eyes. It's one of the hallmarks of the uncanny valley after all.
Well, Photoshop has a series of "Neural Filters" which use AI for various purposes, including one called "Smart Portrait". A feature of this is the ability to alter expressions or change the direction of eye gaze. (I know, I know.)
Running the first photo through this filter and adjusting eye direction ever so slightly creates a more realistic gaze. In addition, by using *another* AI tool (Topaz AI), the image can be upscaled to higher res.
Here's that result.
The main point here is that 3 different AI tools were involved in that last image, and soon some versions of all 3 will all be consolidated into one engine.
posted by jeremias at 3:43 PM on August 5, 2022 [2 favorites]
Anyone remember when it was just one Colab page, in Spanish? VQGAN+CLIP. The old days, last year.
posted by The Half Language Plant at 4:55 PM on August 5, 2022 [3 favorites]
posted by The Half Language Plant at 4:55 PM on August 5, 2022 [3 favorites]
The good: tools that inspire creativity and springboard more ideas, create moods (my RPG group is using these to amazing effect right now).
The bad: AI "upscaling" is an artificial opinion. Based on how humans or other familiar trained visual patterns generally look, it will create an opinion about what should fill in the missing or noisy data. It can be wrong with nearly zero consequences because it's trained with the most satisfyingly likely patterns.
If your great aunt had a surprising scar hidden by that crease in the old photo, the AI is very unlikely* to reproduce it. She will look unscarred, which is great but is only an opinion of how she might look.
* Sometimes the models pick up signals we don't and can use them to make pretty amazing guesses. But not always and not reliably and often that's the very thing you try to tune out... by supplying your own opinion in the form of pruning and training and feature engineering and so forth.
Artificial. Opinion. It's a lovely tool.
posted by abulafa at 5:38 PM on August 5, 2022 [1 favorite]
The bad: AI "upscaling" is an artificial opinion. Based on how humans or other familiar trained visual patterns generally look, it will create an opinion about what should fill in the missing or noisy data. It can be wrong with nearly zero consequences because it's trained with the most satisfyingly likely patterns.
If your great aunt had a surprising scar hidden by that crease in the old photo, the AI is very unlikely* to reproduce it. She will look unscarred, which is great but is only an opinion of how she might look.
* Sometimes the models pick up signals we don't and can use them to make pretty amazing guesses. But not always and not reliably and often that's the very thing you try to tune out... by supplying your own opinion in the form of pruning and training and feature engineering and so forth.
Artificial. Opinion. It's a lovely tool.
posted by abulafa at 5:38 PM on August 5, 2022 [1 favorite]
I played around with some AI image generators and found them occasionally surprising and unsettling in a Francis Bacon sort of way but they seem essentially mediocre to me. I cannot fathom the charm and novelty others find in them. Perhaps it’s because I trained as an artist many years ago and thus corrupted my eye irretrievably. Yes there’s a place for “found” surprises but there is a naive and kitschy coldness in so much of it that misses the mark for me.
posted by Peach at 5:03 AM on August 6, 2022
posted by Peach at 5:03 AM on August 6, 2022
Right now I am just glad that all these tools seem to be explicitly avoiding coming for me; I like drawing cartoon porn and they like censoring the keywords you’d need to use to generate that.
posted by egypturnash at 10:47 AM on August 6, 2022
posted by egypturnash at 10:47 AM on August 6, 2022
The main point here is that 3 different AI tools were involved in that last image, and soon some versions of all 3 will all be consolidated into one engine.I've been thinking a lot recently about how this is the case not just with these images, which are obviously (for now) computer generated, but increasingly for almost every image we see — photos from basically every high-end smartphone for the past many years are passed through many layers of processing, including neural-network based processing.
This computer processing isn't new, of course, it's been normal for decades now, but I think it is pretty new for consumer cameras (and by that, I mean smartphones) to be doing this kind of processing without the user being aware of it at all. And it's also, of course, fairly new to be using NN-based image "correction" algorithms, which even the designers of those algorithms do not fully understand.
posted by wesleyac at 10:09 AM on August 7, 2022
"I played around with some AI image generators and found them occasionally surprising and unsettling in a Francis Bacon sort of way but they seem essentially mediocre to me. I cannot fathom the charm and novelty others find in them. Perhaps it’s because I trained as an artist many years ago and thus corrupted my eye irretrievably. Yes there’s a place for “found” surprises but there is a naive and kitschy coldness in so much of it that misses the mark for me."
Not sure how long ago you played around with these, but the Francis Bacon reference makes me think it refers to the earlier versions of AI output.
The range and quality of images in the last few months has widened considerably, gwint's own examples mentioned earlier in this thread are the perfect example of the next generation of images that have nothing to do with those earlier garish and uncanny ones.
posted by jeremias at 9:09 AM on August 8, 2022 [1 favorite]
Not sure how long ago you played around with these, but the Francis Bacon reference makes me think it refers to the earlier versions of AI output.
The range and quality of images in the last few months has widened considerably, gwint's own examples mentioned earlier in this thread are the perfect example of the next generation of images that have nothing to do with those earlier garish and uncanny ones.
posted by jeremias at 9:09 AM on August 8, 2022 [1 favorite]
Convenience > Quality, the definitive way to predict ALL future media trends, and this is no exception. That said, as someone who has had access to all the best digital visual toys for decades, this stuff absolutely blows my mind, and has jazzed me more than anything I can remember seeing in recent years. Midjourney has occupied my thoughts for a week and a half now, it's deeply addictive like few other things - the first month or two I spent with ArtMatic was an extension of the deep texture editor in Bryce, and the feeling of messing around in MJ really takes me back a quarter century... damn, I'm old. Knowing this is just the beginning of the wave, it's downright intoxicating. Bring on the AI music makers, I'm ready!
posted by dbiedny at 2:33 PM on August 24, 2022
posted by dbiedny at 2:33 PM on August 24, 2022
« Older How to Make a Schadenfreude Schadenfreude Pie | Not a star, a chorizo Newer »
This thread has been archived and is closed to new comments
These seem like great writing prompts and I'm sure we'll have NYMag have a big expose on the first elitist art show that's just AI painting or something stupid like that ("Is it art? Sotheby's auctions first Dall-E painting for $8.5 million NFT bitcoin crypto whatever!"), but I'm more interested in it enhancing creativity in a similar way weed and coffee help writers and artists.
posted by geoff. at 2:07 AM on August 5, 2022 [4 favorites]