"Innovation doesn't happen overnight..."
- Maya Georgieva, Futurist, from her keynote address at Seneca College
AI & Adobe
Article Update: May 6/24
Note: Adobe is not new to AI; we have been incorporating AI functionality into Creative Cloud tools for nearly a decade. The Content-Aware Fill feature in Photoshop CS5, which was released in 2010, is one of the earlier examples.
Let's learn a little about new functionality in Adobe Creative Cloud Desktop and Adobe web tools. We'll look at how Artificial Intelligence is:
- helping in the creative process
- making it easier to tell your story
- helping to iterate and ideate concepts and storyboards
- improving workflows and simplifying complex or repetitive tasks in Photoshop
- enabling quick and professional-sounding podcast content at the click of a button
There's a lot to talk about, and the state of AI at Adobe just keeps getting better. As such, we will also discuss how Adobe is navigating the turbulent seas of Generative AI.
But first, a simple definition of Artificial Intelligence:
Artificial intelligence (AI) refers to the ability of computer systems or machines to perform tasks that typically require human intelligence, such as learning, problem-solving, decision-making, and recognizing patterns. AI uses complex algorithms and statistical models to analyze and process large amounts of data, and it can learn from the data it processes to improve its performance over time. AI has many applications across different industries and is changing the way we live and work.
Source: ChatGPT, when asked, "Give me a simple explanation of artificial intelligence."
WHY is AI important in Education?
I think we can all agree that AI in general is the current talk of the town, but why is it important in education?
In general, Artificial Intelligence has the potential to revolutionize education by making it more personalized, accessible, efficient, and effective. AI can assist educators in their teaching efforts, enhance the learning experience for students, and provide valuable insights for continuous improvement in the field of education.
Overall, text-to-image generative AI can make educational content more engaging, inclusive, and efficient for both students and teachers. It can facilitate better comprehension of complex topics, enhance the teaching and learning process, and promote creativity in education.
Whether it's helping to analyze a long text-based document, or assisting in the communication of an abstract concept, AI can improve the student experience; getting them to the heart of the matter more quickly or testing out a variety of visual ideas to illustrate a topic, without the limitation of visual design skills.
However, it is essential to ensure that the generated images are accurate and contextually relevant to the educational content. Seeing is no longer believing, and students must be taught to be both critical of what they see as well learn to be responsible and ethical in their own use of generative AI.
“Tech gives the quietest student a voice.”
– Jerry Blumengarten, Educator, Consultant, Writer, Speaker
Table of Contents
Responsible innovation cannot be an afterthought.
- Adobe CEO Shantanu Narayan, at Adobe Summit 2024
Generative AI You Can Trust
Content Credentials
Adobe Firefly attaches Content Credentials to assets created or edited using Firefly, indicating that generative AI was used in the creative process. Content Credentials are verifiable details that serve as a digital “nutrition label.” They can show contextual information including an asset’s name, creation date, what digital tools were used and any edits made along the way, helping to bring more transparency to digital content. Powered by free, open-source technology from the Coalition for Content Provenance and Authenticity (C2PA), this data remains associated with the content wherever it is used, published or stored, enabling proper attribution and helping consumers make informed decisions about digital content.
Training Firefly
As part of Adobe’s effort to design Firefly to be commercially safe, we’re training our initial commercial Firefly models on licensed content, such as Adobe Stock, and public domain content where copyright has expired. Additionally, as a founding collaborator of the Content Authenticity Initiative (CAI), Adobe is setting the industry standard for responsible generative AI.
The CAI is a 2000+ strong community of media and tech companies, NGOs, academics, and others working to promote the adoption of an open industry standard for content authenticity and provenance.
Learn More
Generative AI comes to Acrobat
Imagine having the ability to summarize a 10-page - or even 100-page - document in a matter of minutes, and then being able to quickly pull the salient points from that document almost instantly - complete with document citations.
Being able to see a summary quickly means you can get to the most important points faster, and drill down into the document more quickly for the information YOU need.
Welcome to Generative AI in Adobe Acrobat, currently in public beta for individual and teams users of Creative Cloud. Through AI Assistant and Generative Summary, Acrobat can produce document-centered insights, summaries, Q&A's, grounded to the document(s) and complemented with publicly available information. Your digital administrative assistant is ready to help. It can even help you formulate an email that covers those key takeaways!
Functional and Responsible Gen AI
- AI Assistant: AI Assistant recommends questions based on a PDF’s content and answers questions about what’s in the document – all through an intuitive conversational interface.
- Generative summary: Get a quick understanding of the content inside long documents with short overviews in easy-to-read formats.
- Intelligent citations: Adobe’s custom attribution engine and proprietary AI generate citations so customers can easily verify the source of AI Assistant’s answers.
- Easy navigation: Clickable links help customers quickly find what they need in long documents so they can focus their time exploring and actioning the most important information.
- Formatted output: Ask AI Assistant to consolidate and format information into top takeaways, text for emails, presentations, reports and more. A “copy” button makes it easy to cut, paste and pass along.
- Respect for customer data: AI Assistant features in Reader and Acrobat are governed by data security protocols and no customer document content is stored or used for training AI Assistant without their consent.
- Beyond PDF: Customers can use AI Assistant with all kinds of document formats (Word, PowerPoint, meeting transcripts, etc.)
Study Smarter with AI Assistant
Introducing AI Assistant
Sustainable Generative AI
A recent study by nature.com discusses the carbon emissions of AI systems compared to humans in the tasks of writing and illustrating. The study found the use of AI can have a significant impact on reducing the carbon footprint compared to human activities in certain tasks. AI systems, specifically in the tasks of writing and illustration, emit significantly lower amounts of carbon dioxide equivalent (CO2e) compared to their human counterparts.
Adobe Firefly
Firefly? Sensei? What's the Difference?
Firefly falls under the Sensei umbrella, but where Sensei uses machine learning to deepen insights, enhance the creative experience and drive real-time decisions, Firefly is Adobe’s family of new creative generative AI models focused on image generation.
Note: all the images in this web page - except screen grabs - were created using Adobe Firefly.
How is Firefly different from other GEN AI tools?
Designed to be commercially safe
- Trained on Adobe Stock’s 300 million+ high-res, high-value assets, as well as openly-licensed and public domain content.
Integrated workflows
- Firefly Generative AI capabilities are already embedded into other Adobe tools, such as Adobe Express, Photoshop, Illustrator, Adobe Stock and more.
Enterprise-friendly
- Adobe is exploring ways to enable customers to train Firefly with their own creative collateral, generating content in their brand style and design language.
- Firefly is designed to be commercially safe.
- Adobe has a compensation model in place for Adobe Stock contributors whose work is used to train Firefly.
- For commercial customers, Adobe also offers indemnification.
Generative AI for Creators
Firefly is initially focused on the generation of images and text effects, without infringing on anyone's copyright, or IP. Think of Firefly as a creative co-pilot for you and your projects.
Happy Birthday, Firefly!
Adobe Firefly turned 1-year old at the end of March, 2024 and in celebration of that anniversary, a new and powerful feature went live: Structure Reference.
Note: Firefly Model 3 is now available in beta on the Firefly website and in the Photoshop beta.
Generative Expand is now available through the Firefly website as well as the current version of Photoshop and the Photoshop beta.
Firefly currently consists of 6 models:
- Text to Image (Firefly.adobe.com, Photoshop 2024 & Adobe Express)
- Text Effects (Firefly website & Adobe Express)
- Generative Fill (Firefly website & Photoshop 2024) and Generative Expand (Photoshop/Firefly website)
- Generative Recolor (Firefly web site & current Illustrator)
- Text to Vector Graphic (current Illustrator)
- Text to Template (Adobe Express)
Firefly is not trained on images from artists’ portfolios hosted on the Adobe-owned Behance, and does not currently train on any Creative Cloud subscribers’ personal content or content created by enterprise customers.
Adobe has also developed a compensation model for Adobe Stock contributors whose work has been used to train Firefly
Firefly lets you use everyday language in your text prompts to generate extraordinary new content.
Watch Terry White speak on the topic of AI at his Photoshop World 2023 Keynote address.
Promotional reels and a look to the future
MAX 2023 Sneaks
For a taste of where Adobe is going with AI, look no further than the Max Sneaks!
AI & ML (Sensei) in Photoshop
Photoshop has been receiving updates via the public beta for the last several months and now those features are part of the current version of the application.
- Neural Filters - Photo Restoration, Landscape Mixer
- Content-Aware Fill - Sky replacement
- Remove tool (updated)
- Selections and masking (Auto Select Subject, background/sky)
- Photoshop Generative Fill & Generative Expand (LIVE!)
- Contextual Taskbar
The Future is Now - Generative Fill & Generative Expand (Firefly) are integrated into the current Photoshop (V 25.0.0) release!
Videos and demos are below.
You can now realistically extend a scene in any direction by simply using the crop tool. No need to expand the canvas and then make a selection!
Just as the recently released AI masking was the biggest thing to happen to Adobe Lightroom in more than a decade, Generative AI is the biggest thing to happen to Photoshop since the introduction of layers.
You can now realistically extend a scene in any direction by simply using the crop tool. No need to expand the canvas and then make a selection! This will be a huge timesaver when you need to make aspect ratio adjustments to an existing image.
Update Photoshop and try Generative Fill/Expand for yourself!
Content Credentials
As part of Adobe's efforts to meet or exceed the standards of the Content Authenticity Initiative, the latest update to Photoshop also includes the ability to enable Content Credentials, through the Preferences panel.
Neural Filters
Located under the Filters tab in Adobe Photoshop, Neural Filters are a fun and easy way to create compelling adjustments and speed up your image editing workflows. Powered by artificial intelligence and machine learning engine Adobe Sensei, Neural Filters use algorithms to generate new pixels in your photos. This allows you to add nondestructive edits and explore new creative ideas quickly, all while keeping the original image intact.
Photo Restoration Neural Filter
Content Aware ...Anything!!
Fill
The Content-Aware Fill workspace provides an interactive editing experience for ultimate image control. Use the live full-resolution preview as you refine the sampling area Content-Aware Fill uses and adjust the settings to achieve amazing results
Patch
The Patch tool is used to remove unwanted image elements. The Content-Aware option in the Patch tool synthesizes nearby content for seamless blending with the surrounding content.
Move
Use the Content-Aware Move tool to select and move a part of a picture. The image is recomposed, and the hole left behind is filled using matching elements from the picture. You do not need to carry out intensive edits involving layers and complex selections.
Select and Mask
Creating accurate selections and masks in Photoshop is now quicker and easier than ever before. A dedicated new workspace helps you make precise selections and masks. Use tools like the Refine Edge Brush to cleanly separate foreground and background elements and do much more.
The Select and Mask workspace replaces the Refine Edge dialog in earlier versions of Photoshop, offering the same functionality in a streamlined way.
Quickly and easily change a blah sky using built-in images or your own sky photos.
Remove Tool in Photoshop Desktop
The Remove tool replaces areas of your image with realistic and relevant content. Use this tool to remove any distracting regions or elements, no matter how small or large.
Simply brush this artificial-Intelligence-powered tool over objects and people to make them disappear… as if they were never there.
Complicated backgrounds and structures, such as fences or edges are seamlessly completed.
Choose the Remove tool, hover over the item without making a selection, and click!
More Resources
Firefly & Substance 3D
Substance 3D’s latest releases introduce two new Firefly-powered features:
Substance 3D Sampler’s innovative Text to Texture feature offers creators the ability to generate photorealistic or stylized textures for 3D object surfaces from simple text prompts. This feature significantly enhances artists and designers’ creative and iterative processes without the need for physical prototypes, stock imagery or manual photography.
Substance 3D Stager's new Generative Background feature empowers users to create detailed background images from text prompts and seamlessly composite objects into scenes with intelligent perspective and lighting matching.
These new capabilities significantly accelerate the creative review process, making professional design workflows more productive and seamless while saving considerable time, unlocking new realms of creative possibility with new generative workflows designed to complement human imagination, not replace it.
- Quicker ideation
- More creative freedom
- Save time
- Ability to produce:
- High-quality, realistic textures and environments
- High-fidelity visuals and animations to enhance brand presentations and storytelling.
Note: By default, Adobe Firefly attaches Content Credentials to assets created or edited using Firefly, indicating that generative AI was used in the creative process.
Availability
Substance 3D Sampler 4.4 and Stager 3.0 featuring the new Text to Texture and Generative Background features are accessible to all Substance 3D customers starting March 18.
Create the Perfect Podcast
Whether you want to improve the quality of an existing recording, record an interview, or edit a voice audio track, Adobe Podcast can help you out.
Podcast Studio
Powered by Adobe Premiere Pro's speech-to-text technology, Adobe Podcast Studio (beta) lets you easily record, edit, and enhance audio in your browser. Remote recording with others is as easy as sharing a link! Everyone's audio is recorded in high quality locally, and then Adobe Podcast syncs it back together in the cloud automatically.
NEW! Improving the audio quality of a video file is now a snap in Adobe Podcast. Just upload your .mov, Mp4 or .M4V file to the Enhance Speech tool and get studio-quality audio in seconds.
Related Magic
Character Animator (Desktop)
Whether you’re a content creator or animation lover, you can animate a character that’s an extension of you in the new Starter mode.
Select a puppet, record your audio, add motion and emotion, and you’re ready to export an animated character. Starter mode gets anyone animating in just a few minutes.
Character Animator via Adobe Express
Animating has never been easier!
Continue the character creativity inside Express as you pick a puppet, choose a background and add your audio. Then download your video to continue telling your story.
AI in Lightroom
May 2024 Update
Firefly is now part of the Lightroom family as of May 2024, with the introduction of Generative Remove to help you eaisly remove visual distractions in your images.
In addition, Lens Blur (with Bokeh control) has been enhanced to be more accurate and effective
Earlier this year Lightroom also introduced the abilty to edit for HDR displays.
Other AI feature releases
Lightroom Ecosystem
Lightroom Desktop
- Local access & editing (selective Cloud Sync)
- Point Color for precision color editing
Lightroom Mobile
- Redesigned interface for improved user experience
Lightroom Classic
- Point Color for precision color editing
- Filter/search presets
- Performance improvements
Denoise is part of the newer Enhance features of Lightroom Desktop and Lightroom Classic. It was designed to remove noise with the help of artificial intelligence in photographs taken in low light situations that require High ISO settings. This capability will be a boon to photographers shooting with smaller sensor cameras like Micro 4/3 and APSC-sized sensors
AI Masking in Lightroom
You can use new AI-powered masks to automatically select people, objects, and also backgrounds with just a click. You can also use the Mask Amount Slider to adjust several adjustment settings at once
Adobe Illustrator
Create incredible graphics in Illustrator with generative AI.
A beta feature that is NOT in a beta app! Yes, now in the current version of Illustrator 2024 is the new Text to Vector Graphic feature, powered by Adobe Firefly.
Just type a simple description to generate scalable, customizable icons, scenes, and patterns. Plus, use your own artwork as a reference to create on-brand assets for websites, posters, and more.
Generative Recolor
Generative Recolor, uses Adobe Firefly to recolor vector artwork on the Firefly website AND within the current version of Adobe Illustrator.
In Illustrator, simply open a vector graphic, select the object on the canvas and choose Recolor from the Properties Panel, then select Generative Color (Beta) from the Recolor panel. type in any text prompt you want or use one of the samples. I opted for the prompt, "Sunset" for my vector logo
Audio Editing
Audio editing is becoming easier and smarter in Premiere, thanks to AI. One-click voice quality improvements with Enhance Audio, Remix to extend or reduce music tracks and auto-recognition of audio types are just some ways AI makes working easier in Premiere Pro.
Premiere Pro Speech-to-Text
The Captions workspace consists of the Text panel (including the Transcript and Captions tabs). You auto-transcribe your video in the Transcript tab, and then generate your captions, which you can edit in the Captions tab and in the Program Monitor. Captions have their own track on the timeline. Stylize your captions with the design tools in the Essential Graphics panel.
Auto-transcribe on Import
The easiest and fastest way to create captions is to start with a transcript of your video. While importing your media, you can enable Automatic transcription under Import settings.
Text-Based Editing
Text-Based Editing in Premiere Pro is an entirely new way of editing that makes creating rough cuts as simple as copying and pasting text.
Announced at MAX 2023, you can now delete all pauses with a single click, work with multi-channel audio files, and get more control over your transcript view.
AI-based Editing Tips and Tricks
Audio Auto-Tagging
Automatically tag audio files as Dialogue, Music, SFX, or Ambience to reveal controls in the Essential Sound panel to give you immediate access to the most relevant tools to create professional sound.
What's New and Exciting
At Adobe, development never stops! Here is just a sampling of new and/or beta features we're working on.
Firefly lands on Adobe Lightroom PreRelease
Firefly, in the form of Generative Fill, is now part of the Remove Tool options in Lightroom Classic and Lightroom Desktop Prerelease versions. Using a tool you are already familiar with, you now gain added control and realism when you need to remove a distracting element from a photo.
You can download the prerelease versions of Lightroom from the Creative Cloud Desktop App, in the Prerelease tab.
Gen AI Updates to Photoshop Beta
Firefly Model 3:
- Get improved results with Generative Fill and Generative Expand
- Generate Entire image
- Generate variations with a reference image
- Generate a new background
- Enhance Detail - Sharpen the details and clarity of generated variations when using Generative Fill
Firefly comes to Substance 3D (beta)
Adobe continues its innovation momentum with Firefly-powered Creative Cloud upgrades in Substance 3D workflows!
- Adobe launches the first integration of Firefly generative AI capabilities into Substance 3D Sampler and Stager, making 3D workflows more seamless and productive for industrial designers, game developers and VFX professionals.
- Text to Texture and Generative Background features significantly accelerate traditionally time-consuming and intricate 3D texturing and staging tasks.
Text to Image generation in InDesign
The current release of InDesign integrates Firefly into the design workflow. Users can generate graphics in photo and art formats using the same Firefly workflow with the Text to Image panel or new Contextual Taskbar.
Video Captions in Adobe Express
Captioning videos has never been easier! Try out this amazing AI-powered Quick Action in the new version of Adobe Express. Just upload your video and within minutes, your captions are created.
You can choose from various default styles, or take your newly captioned project directly into Adobe Express to further edit the look and feel of your caption effect.
Remove VIDEO Background in Express Beta
The Adobe Express beta now enables you to quickly, easily and accurately remove the background of a video clip. Within seconds you can remove a background and add a new video clip as a replacement
5 steps to remove a video background:
- Find/upload a video to an @adobeExpress beta project
- With video selected, choose Remove Background
- Find Upload a new video background
- Insert background video into project
- Download new video
Adobe Firefly for Video
We are truly in the golden age of video — short-form video is ubiquitous in news, social media, and entertainment. And, insatiable demand, multiplying channels, and globally distributed teams make it extra challenging to scale the production of high-quality creative work efficiently.
This is why we’re excited to invent and innovate with the video and audio community to make it easier and faster for you to transform your vision into reality. With Firefly as a creative co-pilot, you can supercharge your discovery and ideation processes and cut post-production time from days to minutes. And with generative AI integrated directly into your workflows, these powerful new capabilities will be right at your fingertips. Imagine the power to instantly change the time of day of a video, automatically annotate and find relevant b-roll, or create limitless variations of clips — all as a starting point of your creativity.
Coming very soon to Premiere Pro, these revolutionary Generative AI features:
- Generative Extend - add new frames of existing footage to your timeline
- Object Addition and Object Removal
- Generative B-roll
- Integration with 3rd party AI models (Open AI, Runway and Pika Labs)
- Content Credentials for AI-generated footage
Content credentials, will not only understand when AI was used but what model was used in the creation of media.