|
I want to talk about Luma Dream Machine, the tool I used to create the animations I posted recently. First off, it's an incredible tool! They've done an amazing job of solving the flickering issue that most img2vid tools have when handling even basic videos.
However, it is $25 a month for 150 five second videos, and that's a lot of time and money to spend to learn a tool. I think it's worth it, and you should try it out if you like what I've done with it, but it has limitations that I want to tell you about before you spend your time and money finding them out.
One, it doesn't do anatomy well. If you tell it to do any sort of complicated action (or it decides on one itself, see below), it will quite happily rotate heads 360 degrees, bend arms backwards, etc. Maybe this is a skill issue, but (as you can tell) I got the best results from very simple prompts like like "A young British woman talking to the camera. Static camera."
Two, it loves to move that camera. I know they're trying to show off what their software can do, but it does it at the expense of what you're trying to do. Any time I don't tell it "static camera" in the prompt, there's a chance it'll spin like a Matrix movie, wobble around like a drunken sailor, or show your subject for maybe half the video before panning to someone it made up in it's head.
Three, it loves to create new characters. If you mention any sort of interaction, even if you specify that the camera is the target, it'll create a new character out of whole cloth and put it in your video. I tried a prompt like "an angry woman picking a fight with the camera", along with a standard image of mine, and she walked towards the camera for maybe half a second before some guy walked into shot at a hundred miles an hour and they had some sort of anime face off. Don't give it an excuse!
Four, it's good at maintaining anatomy, but the movement is a little fast and unnatural. Also it doesn't really know how long hair works. If you're expecting it to blow in the wind or a prompt like "a woman plays with her hair" to a) work or b) stay consistent with your character, you're out of luck. You might get one, but you'll rarely get both.
All of this isn't to say you shouldn't use the tool. As I said, I love it, but you have to temper your expectations. If you want to see your favourite images in motion, and you just want a TikTok style vid where they talk to the camera, then you can get pretty stunning results pretty quickly. Just don't think you're going to use their img2vid tech to make a Hollywood movie anytime soon.
One last important caveat is to look at the date of this post! AI development is moving insanely fast. I'll try to post more of these as the services I use grow and evolve, but if this ends up being my last post on the matter and you're reading it six months into the future, probably disregard it.
|