Discord Speech Recognition
    Preparing search index...

    Discord Speech Recognition

    Discord Speech Recognition Extension

    This is an extension for discord.js library that makes creating discord speech recognition bots as easy as common text bots.

    Discord.js v14 (Node.js 22.12 or newer):

    npm i discord-speech-recognition discord.js @discordjs/voice @discordjs/opus

    You need one opus library to decode voice. @discordjs/opus is the fastest, but it's a native module. If it doesn't install on your machine, use opusscript instead:

    npm i discord-speech-recognition discord.js @discordjs/voice opusscript

    Checkout simpleBot example in examples directory for ready-to-use bot.

    Discord.js v13:

    npm i discord-speech-recognition@2

    Discord.js v12:

    npm i discord-speech-recognition@1

    https://discordsr.netlify.app/

    const { Client, GatewayIntentBits, Events } = require("discord.js");
    const { joinVoiceChannel } = require("@discordjs/voice");
    const { addSpeechEvent, SpeechEvents } = require("discord-speech-recognition");

    const client = new Client({
      intents: [
        GatewayIntentBits.GuildVoiceStates,
        GatewayIntentBits.GuildMessages,
        GatewayIntentBits.Guilds,
        GatewayIntentBits.MessageContent,
      ],
    });
    addSpeechEvent(client);

    client.on(Events.MessageCreate, (msg) => {
      const voiceChannel = msg.member?.voice.channel;
      if (voiceChannel) {
        joinVoiceChannel({
          channelId: voiceChannel.id,
          guildId: voiceChannel.guild.id,
          adapterCreator: voiceChannel.guild.voiceAdapterCreator,
          selfDeaf: false,
        });
      }
    });

    client.on(SpeechEvents.speech, (msg) => {
      // If bot didn't recognize speech, content will be empty
      if (!msg.content) return;

      msg.author.send(msg.content);
    });

    client.on(Events.ClientReady, () => {
      console.log("Ready!");
    });

    client.login("token");

    You need to enable message content for this example, so it can react to messages in chat.

    The package works the same way with ES modules and TypeScript:

    import { addSpeechEvent, SpeechEvents } from "discord-speech-recognition";

    addSpeechEvent(client, options) adds speech event to the client. The bot listens on every voice channel it joins with joinVoiceChannel, as long as:

    • the client has Guilds and GuildVoiceStates intents,
    • the bot joined with selfDeaf: false, deafened bot doesn't listen to anyone.

    When someone stops speaking, their speech is sent to speech recognition and speech event is emitted with a VoiceMessage:

    Property Description
    content Recognized text, empty if nothing was recognized
    error Error thrown by speech recognition, if there was any
    author, member User who spoke
    channel, guild Voice channel the user spoke on
    duration Duration in seconds
    startTimestamp When the user started speaking, in milliseconds
    audioBuffer PCM mono 16-bit audio with 48kHz sample rate
    connection Voice connection, you can use it to play audio as a reply
    saveToFile(path) Saves audio to .wav file
    toWav() Returns audio as .wav file in a Buffer
    getWavEncodedToBase64Audio() Returns audio as .wav file encoded to base64 
    const speechOptions = addSpeechEvent(client, {
      lang: "pl-PL",
      profanityFilter: false,
    });

    // Options can be changed at any time
    speechOptions.lang = "en-US";
    Option Default Description
    speechRecognition Google Speech Function that turns audio into text, undefined turns speech recognition off
    ignoreBots true Don't listen to other bots
    shouldProcessSpeech Function (user, connection) => boolean, return false to ignore the speech
    minimalVoiceMessageDuration 1 Shorter voice messages are ignored, in seconds
    maximalVoiceMessageDuration 19 Longer speech is split into multiple voice messages, in seconds
    silenceDuration 300 How long the user has to be silent before the voice message ends, in milliseconds
    group Use it if you pass group to joinVoiceChannel

    All other options are passed to the speech recognition function.

    Works out of the box with a generic API key. The key should be used for personal or testing purposes only, as it may be revoked by Google at any time. You can obtain your own key here.

    Option Default Description
    lang en-US Language of the speech
    key Your own API key
    profanityFilter true Replaces vulgar words with asterisks 
    const {
      addSpeechEvent,
      resolveSpeechWithWitai,
    } = require("discord-speech-recognition");

    addSpeechEvent(client, {
      speechRecognition: resolveSpeechWithWitai,
      key: "server access token of your wit.ai app",
    });

    The language is set in the settings of your wit.ai app.

    Speech recognition function receives PCM mono 16-bit audio with 48kHz sample rate and all options passed to addSpeechEvent. It returns recognized text.

    const { addSpeechEvent } = require("discord-speech-recognition");

    async function recognize(audioBuffer, options) {
      const response = await fetch("https://your-speech-api.example", {
        method: "POST",
        headers: { Authorization: `Bearer ${options.apiKey}` },
        body: audioBuffer,
      });
      return (await response.json()).text;
    }

    addSpeechEvent(client, {
      speechRecognition: recognize,
      apiKey: "passed to the recognize function",
      // Raise the limit if your api handles longer audio
      maximalVoiceMessageDuration: 60,
    });

    The function also gets a third parameter with the user who spoke and the voice connection. Most services want a file and not raw audio, pcmToWav converts it:

    const { addSpeechEvent, pcmToWav } = require("discord-speech-recognition");

    async function recognize(audioBuffer, options, { user, connection }) {
      const form = new FormData();
      form.append("file", new Blob([pcmToWav(audioBuffer)]), "speech.wav");
      // ...
    }

    Speech recognition often mishears short names, so accept a few similar words and compare whole words, not parts of the text.

    const wakeWords = ["alexa", "alexia", "alex"];

    client.on(SpeechEvents.speech, (msg) => {
      const words = (msg.content ?? "").toLowerCase().split(/\s+/);
      const wakeWordIndex = words.findIndex((word) => wakeWords.includes(word));
      if (wakeWordIndex === -1) return;

      const [command, ...args] = words.slice(wakeWordIndex + 1);
      if (command === "play") play(msg, args.join(" "));
    });

    shouldProcessSpeech is called when someone starts speaking. If it returns false, the speech isn't sent to speech recognition and speech event isn't emitted.

    let isBusy = false;

    addSpeechEvent(client, {
      shouldProcessSpeech: () => !isBusy,
    });

    client.on(SpeechEvents.speech, async (msg) => {
      if (!msg.content) return;
      isBusy = true;
      try {
        await answer(msg);
      } finally {
        isBusy = false;
      }
    });

    const enabledGuilds = new Set();
    const optedOutUsers = new Set();

    addSpeechEvent(client, {
      shouldProcessSpeech: (user, connection) =>
        enabledGuilds.has(connection.joinConfig.guildId) &&
        !optedOutUsers.has(user.id),
    });

    const {
      addSpeechEvent,
      resolveSpeechWithGoogleSpeechV2,
    } = require("discord-speech-recognition");

    const languages = new Map([["id of the guild", "pl-PL"]]);

    addSpeechEvent(client, {
      speechRecognition: (audioBuffer, options, { connection }) =>
        resolveSpeechWithGoogleSpeechV2(audioBuffer, {
          ...options,
          lang: languages.get(connection.joinConfig.guildId) ?? "en-US",
        }),
    });

    Turn speech recognition off to get the audio without sending it anywhere.

    addSpeechEvent(client, { speechRecognition: undefined });

    client.on(SpeechEvents.speech, (msg) => {
      msg.saveToFile(`./recordings/${msg.author.id}-${msg.startTimestamp}.wav`);
    });

    content is empty both when nobody said anything understandable and when speech recognition failed. Check error to tell them apart.

    client.on(SpeechEvents.speech, (msg) => {
      if (msg.error) console.error("Speech recognition failed", msg.error);
    });

    Libraries like DisTube join the voice channel for you and use id of the bot as the group. This is detected automatically, so addSpeechEvent(client) is enough. For other groups pass group to addSpeechEvent.

    Speech recognition finds the connection only if the music library uses the same copy of @discordjs/voice as your bot. Run npm ls @discordjs/voice, there should be one version in the tree.

    The bot doesn't react to speech

    • Check if the client has Guilds and GuildVoiceStates intents.
    • Check if the bot joined the channel with selfDeaf: false.
    • If you use group in joinVoiceChannel, pass the same group to addSpeechEvent.
    • If a music library joins the channel for you, check "DisTube and other music libraries" above.
    • By default other bots are ignored, set ignoreBots: false to listen to them.
    • Listen to SpeechEvents.audioStreamError event and check msg.error in speech event.

    The bot leaves the channel or doesn't come back after it was disconnected

    This package doesn't manage the voice connection. Handle disconnects the way discord.js guide recommends.

    Recognized text is in the wrong language

    Pass lang option, for example addSpeechEvent(client, { lang: "pl-PL" }).

    npm ci
    npm run lint
    npm run test:unit

    Unit tests work offline. Integration tests use real services: copy .env.example to .env and fill it in to run tests that need two Discord bots and wit.ai key, they are skipped otherwise.

    npm run test:integration

    Settings
    Member Visibility

    On This Page
    Discord Speech Recognition Extension