Unity is joining other companies in making generative AI tools available to its users, but has been careful (unlike some tools) to ensure these tools are built on a solid, non-theft foundation. Muse, the new AI-powered suite of tools, will start with texturing and sprite generation, gradually transitioning to animation and coding as it matures.
The company announced these features as well as the next big version of its cloud-based platform and its engine, Unity 6, at the Unite conference in San Francisco. After months of turmoil— Major product plans completely reversed and CEO ousted — They may be eager to resume business as usual, if possible.
Unity has previously positioned itself as a champion for smaller developers who lack the resources to adopt broader development platforms like rival Unreal. Therefore, the use of AI tools can be seen as a useful supplement for developers who do not have access to AI tools. For example, you could spend a few days creating 32 slightly different high-definition wood wall textures.
While there are many tools to help generate or alter such assets, being able to say “make more of this” without leaving the main development environment is often preferable. The simpler the workflow, the more people can do without having to worry about details like formatting and content. Isolated resources.
AI assets are also often used for prototyping, where things like artifacts and a slightly grainy quality (which are often present regardless of the current model) don’t hold any real importance. However, illustrating your concept gameplay sprites or free sample 3D models with original, appropriate art rather than stock footage can go a long way in communicating your personal vision to publishers or investors.
Examples of sprites and textures generated by Unity’s Muse.
Another new AI feature, Sentis, is a bit hard to understand – Unity’s press release says it “enables developers to bring complex AI data models into the Unity runtime to create new gaming experiences and features.” So it’s a BYO model. Something like that, with some built-in functionality, currently in public beta.
Artificial intelligence for animation and behavior will be added next year. These highly specialized scripting and design processes can benefit greatly from generating first drafts or multiplication assistants.
Image Source: Unite
The Unity team emphasized that an important part of this release is ensuring that these tools do not live in the shadow of upcoming IP infringement cases. Image generators like Stable Diffusion are fun to play with and are built using resources from artists who never consented to their work being ingested and regurgitated.
“To provide useful output that is safe, responsible, and respects the copyrights of other creators, we challenge ourselves to innovate in the training techniques of the AI models that power Muse’s sprite and texture generation,” reads a blog post Responsible AI technology comes with announcements.
The company says it uses a completely custom model that is trained on images owned or licensed by Unity. Although they did use stable diffusion, essentially producing a synthetic larger dataset from the smaller curated datasets they assembled.
Image Source: Unite
For example, this wood wall texture might be rendered in multiple variations and color types using the stable diffusion model, but nothing new is added, or at least that’s how they describe how it works. However, the result is that the new dataset is not only based on reliably sourced data, but also removes a step from it, making it less likely that a specific artist or style will be copied.
This approach is safer, but Unity admits it is of lower quality in the initial models provided. However, as mentioned above, the actual quality of the generated assets is not always very important.
Unity Muse costs $30 per month as a standalone product. No doubt we’ll soon hear from the community about whether this product is worth the money.