Apple - Photos - Memories
“Photos scans your library for significant people, places, holidays, and more, and presents them in curated collections called Memories. Day after day, you’ll find new Memories ready for you to enjoy. You can also create your own Memories, view Memories as slideshows or as automatically edited movies, and share them with friends and family.” (credit: Apple)
Our Role
We were assigned to this project, shepherding it from the discovery phase, through the design and implementation phases. The key tasks during the discovery phase were to define the core problem and distill direction from the executive team. In the design phase, we explored various conceptual directions and ways in which it would fit in the current product line. We also defined the primary user-interface and visual design. The overall project lasted about eighteen months.
Background
The first problem was rooted in the fact that our customers were capturing more photos and videos on their mobile devices. Due to the sheer volume of photos and videos in their Photos libraries, our customers were finding it difficult to look back on their media in a meaningful way.
Early on, our team knew that we wanted to leverage new cutting-edge machine-learning technologies to intelligently group their photos and videos. We looked at the metadata that was collected at the time of capture and decided to organize the media by: important events in their calendars (birthdays, holidays), people that frequently appeared in the photos, and places that were often visited.
Presenting the media in an intelligently grouped manner was not in itself compelling; we wanted to provide a new and interesting way to consume that media. With the team’s background in movie-making software tools, we knew that editing together the best parts of video clips and pairing it with amazing music made for a richer, more compelling storytelling experience.
At the time, Apple offered iMovie and Final Cut Pro as the movie making products for our customers. While iMovie was designed to be a simplified approach to movie making when compared to Final Cut Pro, our customers who never had video editing experience, the learning curve was too high and the amount of time it would take was too long. In short, while we strived to make video editing as easy-to-use as possible, the movie making process required too much time and effort.
The second challenge was to bring that crafted movie making experience to a wider range of our customers. We also acknowledged that through an automatic personal movie experience, it could inspire our customers to advance to our more full-fledged video-editing applications.
Outcome
The Photos app on iOS, iPadOS, and macOS intelligently groups customers photos and videos in more compelling ways than simply by date. When the iPhone, iPad or Mac is plugged in at night, the machine learning algorithm sifts through the customer’s library and groups media based on special events, important people, and frequently visited places. There are also special Memories that are created via creative combination of factors. Some of those memories include: photos from last weekend, memories from this day last year, dinners in the city, breakfast at home, etc.
Photos Memory
There are two concepts in the memories feature in Photos: the Memory and the Memory movie. A Memory is a grouping of photos and videos organized by people, places, events like holidays and birthdays, and the various combinations of those categories.
The Memory movie is a personalized, automatically-created movie that is assembled using the best portions of the photos and videos from that particular memory. The music, duration, editing style, titles, filters, transitions, and layout are all determined by a simple user-interface which we dubbed: the mood dial.
Memory Movie
In the Photos app, available on iOS, iPadOS, and macOS, the Memory is displayed with the Memory movie on top and the photos and videos that comprise the super set of the movie along the bottom. The customer can simply play the movie, or view the individual photos and videos.
Inside the Memory movie, the interface has been designed for absolute simplicity. There are three main controls: the mood dial, duration picker, and the playback controls. The mood dial is initially set to a default position (indicated by the dot), which is what the algorithm has determined is the best setting based on the content featured inside the movie. The particular mood primarily determines what type of music will be paired with the movie, but it also affects the editing style, as more upbeat moods will have more frequent cuts. The duration picker initially had more granularity, but due to the length of music and available content (i.e. photos and videos), the duration was abstracted for simplicity to: short, medium, and long. The playback controls are the same design found during video playback inside the Photos app.
Mood Dial
There are many factors that determine the feeling of a movie. We, as a team, decided that for a Memory movie, those factors would be: music, duration, editing style, titles, filters, transitions, and layout. These factors that are at the heart of the creative editing process is abstracted into a concept we call Mood. When a user picks a particular mood, the app automatically adjusts these seven factors. Of course, there is an option to customize their movie, but the advanced controls are hidden by default and progressively disclosed.
Mood - Happy
Mood - Gentle
Customization
If a customer wants to customize the Memory movie, they can tap on the Edit button from the Memory movie. Through the approach of progressive disclosure, more advanced and complex features are available as needed, but do not overwhelm a customer who has no desire to see or use them.
The customer has the ability to custom pick the song used for the movie and the app automatically cuts to the beat of the song. They can also add or remove any individual photo or video, and trim (pick the in and out edit point) for any individual video clip.
A Starting point to learn Filmmaking
With Memory movies and the customization feature, we viewed this as a great starting point for customers who have an interest in learning video editing and filmmaking. Every Memory movie can be opened in iMovie or Final Cut Pro X. Every video clip, music, and title object shows up as an iMovie or Final Cut Pro X project as expected.