What is the Sequins Web Prototype?

Sequins is a web-based prototype that I used to validate a concept for a hardware-based electronic musical instrument. I am more familiar with web technologies than with embedded technologies and I knew I could iterate more quickly on my ideas this way. After successfully building out this prototype and validating the original idea, I moved on to developing the hardware version. Read on to learn about the process of developing this web-based version!

Clickity-clack inspiration

A few years ago I was starting to dip my toes into the world of mechanical keyboards, and I put together an olkb planck kit. The Planck distinguishes itself from the competition through its uncommon ortholinear design, which resembles a perfect grid of keys.

I was pressing the keys and enjoying the tactility of the mechanical switches when I thought to myself, "wouldn't it be cool to have a looper with a 16 step sequencer interface with which to chop up and re-sequence your loops?". At the time, this was just a fuzzy idea, and I had no idea where to even begin. Little did I know this daydream would spark a years-long journey researching electrical engineering concepts and embedded systems design.

A Whole New World

After months of obsessively reading through every possible path related to electricity on Wikipedia, I decided to properly dive in. I invested in an oscilloscope, a function generator, a variable power supply and a multimeter and started working through The Art of Electronics, only to discover that I need to brush up on my calculus before I can truly grok the deep EE stuff. But that did not deter me! I learned what I could about all the basic electrical components, their purposes, and how to put them to use. I decided to take that newfound knowledge and apply it to a project, and I designed and built my very own Fuzz Pedal for my guitar, which was a challenging yet extremely rewarding feat.

With that notch in my belt I decided to return to my original idea, but after experiencing firsthand the complexity of engineering and debugging physical electronic circuits and microcontroller firmware, I decided it would be wiser to start somewhere more familiar: the web! So I embarked on another multi-month quest to iterate on this yet-unhatched idea.

Humble Origins

The project began as pure javascript within an html file. As this was meant to be a prototype for a playable musical instrument, I started with the most fundamental features. In the video below you can see an audio sample displayed as a waveform, sliced into multi-colored 'chops'. Each chop is assigned to a key on the user's keyboard. Pressing a key triggers playback of its associated chop's portion of the original sample, so the user can quickly get a feel for the 'piano roll' that they have generated by manipulating the below controls. The first range input allow the user to specify the number of chops to slice the sample into, which can drastically change the feel of the piano roll (depending on the sample). The second input adjusts the length of each chop, so that the user can control the degree to which the chops sound staccato or legato.

The result was an ugly duckling that was instantly satisfying and fun to play with! I was inspired to continue and immediately set off developing the next feature, a sequencer that would allow users to assign individual chops to a step in a sequence and play back in a loop. I also added a range slider to control the tempo, as well as the number of steps in the sequence so that it's not confined to the standard 8 bars.

Becoming its own thing entirely

It was about here that I forgot that I was making a prototype and stumbled into developing this beyond the scope of my original intention. In the context of the hardware looper/sequencer prototype, I hadn't really imagined supporting multiple tracks. However, on a computer, with access to a whole internet of sounds, it just felt limiting to confine the user to a single track. So, I re-designed the UI to package all the various controls into a per-sample container.

Multi-track capable

When it finally came time to add an additional sample into the mix, it was time to kiss my custom js framework goodbye. I'd worked with React in the past, and I realized that I had re-invented the wheel how I was handling state and renders. Besides, it had already been unwieldy working out of an html file importing javascript files directly for a while. So, I spun up a node app and re-built everything in React. I was sad to see my custom system go, but I was also excited to work in React on a non-work-related project, and to get to explore the new functional components and hooks features. The switch went swimmingly and as a result, adding a new track was a breeze.

Making it beautiful

At this point I decided to commit to transitioning from a hacky prototype to a proper web experience. I refined the color pallete and decided to give the controls some love. I had been using an open-source plugin for rendering the range controls as circular knobs, which gave it a skeumorphic vibe befitting the original intention, but ultimately that user experience fell short in a lot of ways. I wanted a control that could be used with finesse and that would be easy to provide fixed values to. In addition, I wanted it to match the square motif that I had inadvertently established in the design language. It wasn't easy and took a lot of trial and error, math, and in-depth React research-turned-knowledge to get it right, but I'm extremely happy with the end result.

From Instrument to DAW

So now we could upload multiple samples, slice them each up into chops, sequence their respective chops, and modify all sorts of parameters in real time as we loop the sequences. It's all really fun, but as a musician, the feature that I felt was sorely missing is an arrangement, or song view. I didn't just want to play this thing live, I wanted to use it to make songs! So I got to work building out a grid to represent the 'Song View'. I originally tried to do this with divs, but this presented an optimization issue as the number of divs multiplied when I added more beats or samples to the song. I took a deep dive into html canvas and requestAnimationFrame and wrote my own renderer.

The arrangement view opened up a whole new world of possibilties, but it made me realize that another crucial feature was missing: The ability to make multiple sequences for each sample! So, I added a 'Patterns' bar above each waveform that would display the filename of the sample, as well as the list of patterns that you've created from it. Each pattern button includes a colored-block visual representation of the sequence it contains so you can tell them apart at a glance. Similarly, the blocks in the arrangement view split each beat into the same colored-block representation. In the videos below, you can easily spot the difference between the four-on-the-floor kick pattern and the double-time kick sample.

To Be Continued

It was around this time that COVID 19 hit, and I was among the first to get sick. I retreated to CT for the first 6 months of the lockdown, and put down this project while I recovered. When I came back to NYC in the fall of 2021, I would take what I had learned here and begin work on the Sequins Hardware Prototype. Feel free to give Sequins a try.