How Microsoft Surface and the App StaffPad are empowering music creativity

Lead photo: Eleven-year-old composer Garrett Weyenberg, right, and StaffPad creator David William Hearn are pictured in Angel Studios in London on Jan. 29, 2016. Photo by David Palmer.

From Microsoft News:

The songs began when Garrett Weyenberg was 2 and would climb onto a piano bench and make up little tunes, each an ode to things like snowmen or peaches. By ages 5 and 6, he was composing full, intricate songs, sometimes scooting his entire body along the piano to play multiple octaves.

By age 10 last year, Garrett finished an eight-minute piece called “Sonatina in G Major,” a lively work in three movements that won a regional composition contest. It was just one song in a vast oeuvre of original music, built from a lifetime of melodies flowing from Garrett’s head. But until recently, he had no easy way to write his musical thoughts, no practical means of saving his music and sharing it with others to play.

That all changed when Garrett turned 11. He got a Surface Pro 3 for his birthday and started using StaffPad, a notation app that lets users handwrite music and save it for editing, playback and sharing. Designed for Microsoft Surface, the groundbreaking app uses the device’s pen-and-touch technology for a natural, tactile experience, and combines it with modern tools in handwriting recognition, orchestral playback and score editing.

“I like that it’s really simple and intuitive. I can just jot down ideas and it’s easy for me to get a song down now,” said Garrett from his home in McKinney, Texas. He can now write music wherever he is – at a piano or in the car – and his scores are saved in OneDrive for seamless syncing and access across devices.

Garrett recently flew to London to meet the creator of StaffPad, David William Hearn, a composer and music producer known for his work in TV and movies, from “Friday Night Lights” to “Les Misérables.” After Garrett’s mom emailed Hearn about StaffPad, the meeting arose as a way for Garrett to meet the man behind his favorite app, and for the maker to meet a young musician empowered by his creation. The two worked on one of Garrett’s songs in a session filmed for a new video by Microsoft.

“He’s obviously a very talented kid, very musical,” said Hearn, who’s based in London. He’s thrilled that StaffPad, available exclusively in the Windows Store, enables creativity in musicians around the world, from kids like Garrett to retired hobbyists to professional composers.

“It’s amazing how far-reaching music is,” he said. “There’s so many people writing music, so many people creating, which is just such a lovely thing. It goes so far beyond what I thought we could achieve with the app.”

Hearn was inspired to create StaffPad while working on a film score and wishing there was a way to make quick edits with other musicians, instead of having to go back to his studio and fire up his computer. He also wanted to return to the ease of handwriting music and evolve it with technology.

“There’s nothing more natural and fluid than picking up a pen and writing down your musical thoughts,” said Hearn. “It was really an idea of how we could push written music forward.”

StaffPad converts handwritten notes into a beautiful typeset score, with no mouse, keyboard or complicated notation software to fiddle with. The app’s simple interface enables musicians to focus on music and not technology, and Surface tablets deliver a smooth, mobile and productive experience.

“Before StaffPad, I was really just sort of chained to the desk,” said Hearn. “It’s great to have the option to move away from the computer and get outside, or sit down at the piano and just write as you would on paper.”

Recently updated for Windows 10, StaffPad now has a handy “Composer Assistant” that knows voice commands to add woodwinds or change the key signature. And Microsoft’s Universal Windows Platform means StaffPad works on the 84-inch Surface Hub, a great educational tool Hearn is using with the Royal Academy of Music in London on new ways to teach music in a lecture hall.

Before Garrett had StaffPad, his mom used to record hours of video of him playing piano, so that he could later score his music. But notating was so tedious that it rarely happened. It took Garrett a year to write his seven-page “Sonatina in G Major,” a feat he said would now take a week or two with StaffPad and Surface.

“He is composing music in his mind daily,” said Garrett’s mother, Stephanie Weyenberg. “StaffPad truly has been the game changer for him.”

When a medical condition left Garrett in excruciating pain in recent weeks, writing music was an emotional balm, and the technology enabled him to write from bed. It’s helped deepen his love for music as a form of expression and a way to connect with others.

“It’s kind of cool for me that other people could play something that originally I was thinking,” he said.

For Microsoft, StaffPad highlights the power of technology – from natural touch input to Surface Pen precision —to help people young and old express creativity in new ways.

“We’re so inspired by Garrett’s passion and use of StaffPad’s powerful capabilities, the way David envisioned for every professional, educator, student or individual,” said Mona Cao, product marketing manager for Surface.

“It’s digital creation at its best.”

Story by:  Vanessa Ho, Microsoft News Center Staff

Microsoft Fetch! guesses the breed of your dog — or what breed you are.


Microsoft Garage recently released a fun new iPhone App — Fetch!

It’s Joey and Fletcher Approved — it correctly identified their breeds even though I purposely took *terrible* pictures of them. Their faces were mostly covered as they slept. I haven’t had the nerve to find out if I belong in their “pack.” 🙂

From the Microsoft Press Release:

Man’s best friend has inspired a new app – Fetch! Using your iPhone camera or photo library, it can identify and classify dogs by breeds and tell you what kind of human personality fits best with specific breeds. And just for fun, the app will even take an informed guess on what kind of dog you or your friends might be.

Released through the Microsoft Garage just in time for the American Kennel Club’s Meet & Compete and the Westminster Kennel Club Dog Show, this mobile app demonstrates the potential for Microsoft  researchers’ continued advances in artificial intelligence, which have already appeared in other playful ways through Microsoft Project Oxford-powered experiences such as and Mimicker Alarm. In Fetch!, Project Oxford works together with some powerful new machine learning technology to deliver interesting results for all kinds of photos.

“There was an interest in creating a framework that would allow you to take a domain – in our case, dogs – and recognize numerous classes, such as breeds. We were interested in enabling an app to allow you to make object recognition extraordinary, fun and surprising,” says Mitch Goldberg, a development director at Microsoft Research whose Cambridge, U.K based team built the experience. His team works at the intersection of user experience, machine learning, computer vision and more recently, intelligent cloud services. He’s also had two German shepherd dogs, though now he has a cat. “We wanted to bring artificial intelligence to the canine world. We wanted to show that object recognition is something anyone could understand and interact with.”

Fetch! is designed for repeat use, and after giving it a couple tries, it’s easy to see how addictive it can be. You start with your dog, or your friends’ dogs. If the dog’s breed is unknown, the app will show a percentage of the closest breed. Tapping the percentage rosette leads to the top five breeds that could be in the dog. Clicking on the arrow in the corner leads you to more information on the breed.

“If you want to take photos of dogs, it will tell you what dog breed it is, if it’s one of our supported breeds,” Goldberg says. “If I choose to take a photograph of a flower, it’ll say, ‘No dogs found! Hmmm… This looks more like…flower?’ But if you take a picture of a person, it’ll kick into its hidden fun mode. And in a playful way, it’ll communicate to you not only what type of dog it thinks you are, but also why. It’s fun to see if the app knows it’s not a dog. A lot of the time, it’ll tell you what that image is. When there’s not a dog, you still want to use it.”

No two pictures yield the same result. You could resemble a Doberman Pinscher in one photo (sunglasses, no makeup) or a Pekingese (no glasses, makeup) in another. If you photograph an inanimate object, it might tell you, “No dogs found!” and make an informed guess at what it is.

If you like what you see, you can share the image on your social networks and through email.


Apple has biggest App Store season ever

Apple announced that it had its best ever sales this past holiday season. App store and in-app purchases topped 1.1 Billion dollars (cue Dr. Evil’s pinky finger) in a scant two week timeframe ending January 3, 2016.

“The App Store had a holiday season for the record books. We are excited that our customers downloaded and enjoyed so many incredible apps for iPhone, iPad, Mac, Apple Watch and Apple TV, spending over $20 billion on the App Store last year alone,” said Philip Schiller, Apple’s senior vice president of Worldwide Marketing. “We’re grateful to all the developers who have created the most innovative and exciting apps in the world for our customers. We can’t wait for what’s to come in 2016.”