🗣️ Pink Trombone - Bare-handed Speech Synthesis

Sound is generated in the glottis (at the bottom left), then filtered by the shape of the vocal tract. The voicebox controls the pitch and intensity of the initial sound - Neil Thapen

🗣️ Pink Trombone - Bare-handed Speech Synthesis

A programmable version of Neil Thapen's famous and wonderful Pink Trombone

📚 Table of Contents

📦 Setting Up

👄 Producing Sound

👀 Enabling and Disabling the UI

🎛️ Audio Parameters

🎺 Manipulating Vocal Tract Constrictions

📦 Setting Up

Save a local copy of pink-trombone.min.js and pink-trombone-worklet-processor.min.js and make sure they're both in the same relative location (the first will import the other as a Audio Worklet Processor)
In your HTML <head></head> element, insert the file in a script element as a module:

<script src="pink-trombone.min.js" type="module"></script>

In your HTML <body></body> element, insert the following custom element:

<pink-trombone></pink-trombone>

In your JavaScript code, grab the <pink-trombone></pink-trombone> element:

var pinkTromboneElement = document.querySelector("pink-trombone");

Add a load eventListener to the pinkTromboneElement element:

pinkTromboneElement.addEventListener("load", myCallback);

In the "load" callback, assign an Audio Context using .setAudioContext(myAudioContext) (if none is specified, an Audio Context instance is created for you):

function myCallback(event) {
  pinkTromboneElement.setAudioContext(myAudioContext)
}

This method returns a Promise once the AudioWorkletProcessor module is loaded.

In the promise resolution, a Pink Trombone audio node is created, which you can connect to other audio nodes from the scope of the <pink-trombone></pink-trombone> element:

function myCallback(event) {
  pinkTromboneElement.setAudioContext(myAudioContext)
    .then(() => {
      const audioContext = pinkTromboneElement.audioContext
      pinkTromboneElement.connect(audioContext.destination);
    });
}

👄 Producing Sound

😃 To start generating sound, run the .start() method:

pinkTromboneElement.start();

🤐 To stop generating sound, run the .stop() method:

pinkTromboneElement.stop();

👀 Enabling and Disabling the UI

🙂 To show the interactive visualization:

pinkTromboneElement.enableUI();

✍️ To start animating the visualization:

pinkTromboneElement.startUI();

🛑 To stop animating the visualization:

pinkTromboneElement.stopUI();

😊 To hide the interactive visualization:

pinkTromboneElement.disableUI();

🎛️ Audio Parameters

The audio parameters of the Pink Trombone audio node can be accessed from the <pink-trombone></pink-trombone> element's scope:

🎚️ Intensity

pinkTromboneElement.intensity;

🎵 Frequency

pinkTromboneElement.frequency;

👄 Tenseness

pinkTromboneElement.tenseness;

📢 Loudness

pinkTromboneElement.loudness;

〰️ Vibrato

pinkTromboneElement.vibrato.frequency;
pinkTromboneElement.vibrato.gain;
pinkTromboneElement.vibrato.wobble;

👅 Tongue

// 'index' and 'diameter' refer to the tongue's location in the mouth
pinkTromboneElement.tongue.index;
pinkTromboneElement.tongue.diameter;

To change the voiceness between voiced and voiceless, change the .tenseness and .loudness audio parameters as follows:

function setVoiceness(voiceness) {
  const tenseness = 1 - Math.cos((voiceness) * Math.PI * 0.5);
  const loudness = Math.pow(tenseness, 0.25);
  
  pinkTromboneElement.tenseness.value = tenseness;
  pinkTromboneElement.loudness.value = loudness;
}

// voiced
setVoiceness(1);

// voiceless
setVoiceness(0);

Later on I may add a .voiceness audio parameter that automates this - for now I'm just adopting the original version

🎺 Manipulating Vocal Tract Constrictions

Vocal Tract constrictions comprise of an object containing .index and .diameter Audio Parameter properties that are implicitly connected to the Pink Trombone audio node

To add a vocal tract constriction:

var myConstriction = pinkTromboneElement.newConstriction(indexValue, diameterValue);

To set a vocal tract constriction:

myConstriction.index.value = newIndexValue;
myConstriction.diameter.value = newDiameterValue;

To remove a vocal tract constriction:

pinkTromboneElement.removeConstriction(myConstriction);

👅 Common Phonemes

For reference, here are some preset index & diameter preset values for some phonemes:

👅 Tongue phonemes:

æ [pat]

index : 14.93
diameter : 2.78

ɑ [part]

index : 2.3
diameter : 12.75

ɒ [pot]

index : 12
diameter : 2.05

ɔ [port (rounded)]

index : 17.7
diameter : 2.05

ɪ [pit]

index : 26.11
diameter : 2.87

i [peat]

index : 27.2
diameter : 2.2

e [pet]

index : 19.4
diameter : 3.43

ʌ [put]

index : 17.8
diameter : 2.46

u [poot (rounded)]

index : 22.8
diameter : 2.05

ə [pert]

index : 20.7
diameter : 2.8

🎺 Vocal Tract Constriction phonemes: voiced and voiceless consonants share the same values, differing in voiceness

Fricatives
- (ʒ, ʃ) ["s" in "pleasure"]
  - index : 31
  - diameter : 0.6
- (z, s) ["z" in "zoo"]
  - index : 36
  - diameter : 0.6
- (v, f) ["v" in "very"]
  - index : 41
  - diameter : 0.5
Stops
- (g, k) ["g" in "go"]
  - index : 20
  - diameter : 0
- (d, t) ["d" in "den"]
  - index : 36
  - diameter : 0
- (b, p) ["b" in "bad"]
  - index : 41
  - diameter : 0
Nasals
- (ŋ) ["ng" in "hang"]
  - index : 20
  - diameter : -1
- (n) ["n" in "not"]
  - index : 36
  - diameter : -1
- (m) ["m" in "woman"]
  - index : 41
  - diameter : -1

🏆 Developer Showcase

Send us an email at zack@ukaton.com if you have a cool application made with our api!

🙏 Developer Wishlist

Our time is limited, so we'd greatly appreciate it if you guys could implement some of these ideas:

IPA Speak n' See 🗣️💬 - Take input speech from the user using the Media Recording API and approximate their articulation using the Pink Trombone, allowing speakers to visualize how they speak.
Phonetic Voice Editor 🎹👄⌨️ - Create a cross between a Text Editor and a Digita Audio Workstation, where the user can type in phonemes instead of characters, with automation to programmatically adjust the cadence, pitch, and other features over time.
SSML Simulator 📝💬 - Implement a Speech Synthesis Markup Language emulator that can take an utterance and process the speech request using Pink Trombone's audio processing

Name		Name	Last commit message	Last commit date
Latest commit History 57 Commits
.vscode		.vscode
developerShowcase		developerShowcase
images		images
script		script
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
index.html		index.html
package-lock.json		package-lock.json
package.json		package.json
pink-trombone-worklet-processor.min.js		pink-trombone-worklet-processor.min.js
pink-trombone.min.js		pink-trombone.min.js
rollup.config.js		rollup.config.js

License

zakaton/Pink-Trombone

Folders and files

Latest commit

History

Repository files navigation

🗣️ Pink Trombone - Bare-handed Speech Synthesis

📚 Table of Contents

📦 Setting Up

👄 Producing Sound

👀 Enabling and Disabling the UI

🎛️ Audio Parameters

🎺 Manipulating Vocal Tract Constrictions

👅 Common Phonemes

🏆 Developer Showcase

🙏 Developer Wishlist

📖 Bibliography

About

Topics

Resources

License

Stars

Watchers

Forks

Languages