<p>I asked a question in the similar topic couples of weeks ago:
<a href="https://bioacoustics.stackexchange.com/questions/827/annotate-acoustic-and-visual-events-of-a-multimodal-signal/938#938">Annotate acoustic and visual events of a multimodal signal</a>.</p>
<p>To get real good quality scrolling spectrogram in video I use <code>dynaSpec package</code> in <code>R</code> that allows you to generate MP4 video of a spectrogram: <a href="https://marce10.github.io/dynaSpec/" rel="nofollow noreferrer">https://marce10.github.io/dynaSpec/</a>.</p>
<p>When I do a video recording of a vocalizing animal and I want to review both the visual behaviour and the spectrogram of the vocalization, I first create a scrolling spectrogram using <code>dynaSpec</code>. Then, I combine the video recording with the scrolling spectrogram using Davinci Resolve software (free version).</p>

CC BY-SA 4.0
2022-08-15T15:39:22.733Z
This is a great answer to the second option, which is creating a scrolling video and adding it with video editing software. I'd prefer some software that just loaded the video and played a spectrogram in real-time. But your solution to the creation of scrolling spectrograms is a definite improvement over the way I create spectrogram movies. Right now I'm stuck screen recording. Now all I have to do is learn R.
Marc D
CC BY-SA 4.0
2022-08-18T20:23:39.083Z
I have a collaborator trying to make this work but he's running into difficulties. I'll report back if he gets it up and running. Meanwhile, I'm just screen recording Raven windows so I can move forward. That may end up being the easiest solution that is accessible to me after all.
Marc D
CC BY-SA 4.0
2022-08-19T15:23:19.810Z
My collaborator got the R package working, but it is painfully slow. He did a three-second file and it took 3 minutes. We set up an hour-long file and let it go overnight. It says it still has more than 7 days. Does this jive with anyone else's experience with this package? Or is something wrong on our side?
Marc D
CC BY-SA 4.0
2022-08-22T21:46:59.987Z
Hi @MarcD , I haven't used this package to date, but R can be notoriously slow with some (seemingly) straightforward processes. I'm assuming you're using a high performance computer to implement the package?
Tristan Louth-Robins
CC BY-SA 4.0
2022-08-23T09:42:28.157Z
Hello, it is right that R is very slow to create such MP4 file. I think the package was created just for making a short video of scrolling spectrogram. Even with my gaming computer that is very powerfull, it takes ages to create scrolling spectro, while creating long videos with Davinci Resolve as example, is much faster.
Mathieu Mahamoud Issa
CC BY-SA 4.0
2022-08-15T11:50:14.440Z
2022-10-19T12:13:55.630Z
2022-10-19T12:13:55.630Z
Mathieu Mahamoud Issa
<p>There is a similar question on <a href="https://video.stackexchange.com/questions/23222/ffplay-is-there-a-way-to-display-a-video-and-spectrogram-in-the-same-output">this Video SE</a>.
One suggestion/answer uses ffplay (linux?) to combine video with scrolling spectrogram.</p>

CC BY-SA 4.0
2022-08-15T15:41:04.043Z
Thank you. I wish I could try that but I think this method is technically out of reach for me. No Linux knowledge.
Marc D
CC BY-SA 4.0
2022-08-13T09:33:05.773Z
2022-08-13T09:33:05.773Z
WMXZ
<p>I’m not sure that this will fully solve your problem, but for the alternative you have listed (generate scrolling spectrogram + overlay w/ video in adobe premiere), <a href="https://ffmpeg.org/download.html" rel="nofollow noreferrer">ffmpeg</a> is a useful tool to consider. The “<a href="https://ffmpeg.org/ffmpeg-filters.html#showspectrum-1" rel="nofollow noreferrer">showspectrum</a>” function from ffmpeg is a great option for producing scrolling spectrogram videos from a .wav by running a (relatively) simple shell script. This tool gives you lots of flexibility with spectrogram parameters, aesthetics, etc. (<a href="https://keunwoochoi.wordpress.com/2019/07/18/generate-a-spectrogram-scrolling-video-of-an-audio-file/" rel="nofollow noreferrer">a good example is found here</a>). You can create a similar .sh script for your needs and run from the command line to generate your scrolling spectrogram video (or many such videos) without screen recording.</p>
<p>This of course would still require syncing with video, but will at least automate and greatly speed up your generation of scrolling spectrogram videos. As another commenter suggested, dynaSpec (R package) also appears to be a useful tool for this purpose but I do not personally have experience working with that package.</p>

CC BY-SA 4.0
2022-08-18T20:15:16.617Z
Thank you. It's not clear to me how much time that would save me vs. screen recording, especially as I'd need to learn ffmpeg first. I think I'm just going to record the Raven window. It is realtime, but with my long recordings, I can just let it run. The screen recorder I have can just stop after a set period of time, so I don't have to stick around. I may look back at that option when I have more data, but for a few days worth of recording, I'll probably just go with what I know.
Marc D
CC BY-SA 4.0
2022-08-18T03:28:26.797Z
2022-08-18T03:28:26.797Z
Will Oestreich
<p>I do not know how adaptable this set of tools is beyond the specific CATS animal-borne tagging application, but this paper provides a set of code/tools that pairs video, acoustic, and accelerometer data:</p>
<p><a href="https://animalbiotelemetry.biomedcentral.com/track/pdf/10.1186/s40317-021-00256-w.pdf" rel="nofollow noreferrer">https://animalbiotelemetry.biomedcentral.com/track/pdf/10.1186/s40317-021-00256-w.pdf</a></p>
<p>The paper points to the GitHub repositories for the code and a workshop with tutorials and wikis to guide new users.</p>

CC BY-SA 4.0
2022-08-18T20:16:38.983Z
Wow! That looks powerful and interesting. It could be overkill and would be a steep curve for me. Maybe for the next phase of my project where I hope to have more data and some little more technical support who could pull it together. Thanks.
Marc D
CC BY-SA 4.0
2022-08-17T19:29:30.557Z
2022-08-17T19:29:30.557Z
selene
<p>If you end up working with any of the tools in the GitHub repository mentioned by Selene and run into any problems/questions, let me know and we can work through them together!</p>

CC BY-SA 4.0
2022-08-18T20:18:37.640Z
Thanks so much for the offer. It looks like an amazing tool. I wonder if it is overkill given that I only have video and spectrograms. Also, if it would handle the 5.7 k video. Anyway, I think it is too much for me to pull off at this stage. It's a preliminary study with limited hours of data. I think I'm going to bite the bullet and screen record. Maybe in the next phase with more data.
Marc D
CC BY-SA 4.0
2022-08-18T15:55:49.820Z
2022-08-18T15:55:49.820Z
William Gough
<p>I’m looking for a way to review very high resolution video alongside a scrolling spectrogram of the associated audio track. Some DAWs that have spectrograms (Adobe Audition,Reaper) allow for simultaneous video but the video won’t playback at full frame rate or resolution. VLC can handle the video and has audio visualizations, but not a spectrogram. Anyone know of a tool?</p>
<p>It isn’t enough to route the audio to a separate program running a real time display. I need to be able to scroll around the video and see the spectrogram as I do.</p>
<p>As an alternative, I’ve been making a movie of a spectrogram using screen recording, then overlaying this on the video in Adobe Premiere. This actually works very well for playback but it’s a pain to make the movies. I just have to screen record them and manually sync them up. Surely someone has a tool that can generate a video of a scrolling spectrogram without screen recording  and faster than real-time. Thoughts?</p>

CC BY-SA 4.0
2022-08-18T20:21:50.337Z
For anyone else who's interested, Boris [link] (http://www.boris.unito.it) looks like a very user-friendly way to do this and it seems it would handle the high-res video smoothly. Sadly they don't support Mac anymore so I'm unable to make it work. Linux or Windows, it might be the answer.
Marc D
CC BY-SA 4.0
2022-08-25T16:41:06.657Z
I edited the title of your question a bit to make it more detailed/specific and better fit the SE Q/A format. I hope to have maintained the essence of your question, but if not, please comment here or edit back. For more info see: [How do I write a good title?](https://meta.stackexchange.com/questions/10647/how-do-i-write-a-good-title)
selene
CC BY-SA 4.0
2022-08-25T16:44:48.800Z
sometimes comments can be deleted over time or get hidden, your suggestion would make a great answer for others looking to solve the same problem. You are welcome to answer your own question on SE, so would you mind turning your above comment about Boris into an answer?
selene
CC BY-SA 4.0
2022-08-26T17:34:29.770Z
Happy to do so but how do I change the comment into an answer? Do I repost it as an answer?
Marc D
CC BY-SA 4.0
2022-08-26T17:52:39.600Z
yup exactly...You can just scroll down to the bottom and add it as an answer (there should be a white answer box you can type in or a blue button that says "answer your own question" or something like that. Then if folks try it and it helps them, they can vote for it!
selene
<methods><spectrogram-annotation>
How can I pair a scrolling spectrogram with high resolution video?
# How can I pair a scrolling spectrogram with high resolution video?

(Asked by: username_0 on 2022-08-13)

I’m looking for a way to review very high resolution video alongside a scrolling spectrogram of the associated audio track. Some DAWs that have spectrograms (Adobe Audition,Reaper) allow for simultaneous video but the video won’t playback at full frame rate or resolution. VLC can handle the video and has audio visualizations, but not a spectrogram. Anyone know of a tool?

It isn’t enough to route the audio to a separate program running a real time display. I need to be able to scroll around the video and see the spectrogram as I do.

As an alternative, I’ve been making a movie of a spectrogram using screen recording, then overlaying this on the video in Adobe Premiere. This actually works very well for playback but it’s a pain to make the movies. I just have to screen record them and manually sync them up. Surely someone has a tool that can generate a video of a scrolling spectrogram without screen recording and faster than real-time. Thoughts?

**Tags:** methods, spectrogram-annotation

> **Comments:**
> **username_0**: For anyone else who's interested, Boris [link] (http://www.boris.unito.it) looks like a very user-friendly way to do this and it seems it would handle the high-res video smoothly. Sadly they don't support Mac anymore so I'm unable to make it work. Linux or Windows, it might be the answer.  
> **username_1**: I edited the title of your question a bit to make it more detailed/specific and better fit the SE Q/A format. I hope to have maintained the essence of your question, but if not, please comment here or edit back. For more info see: [How do I write a good title?](https://meta.stackexchange.com/questions/10647/how-do-i-write-a-good-title)  
> **username_1**: sometimes comments can be deleted over time or get hidden, your suggestion would make a great answer for others looking to solve the same problem. You are welcome to answer your own question on SE, so would you mind turning your above comment about Boris into an answer?  
> **username_0**: Happy to do so but how do I change the comment into an answer? Do I repost it as an answer?  
> **username_1**: yup exactly...You can just scroll down to the bottom and add it as an answer (there should be a white answer box you can type in or a blue button that says "answer your own question" or something like that. Then if folks try it and it helps them, they can vote for it!  

---

## 5 answers

### Answer by username_2 on 2022-08-15. Score: 6

I asked a question in the similar topic couples of weeks ago:
[Annotate acoustic and visual events of a multimodal signal](https://bioacoustics.stackexchange.com/questions/827/annotate-acoustic-and-visual-events-of-a-multimodal-signal/938#938).

To get real good quality scrolling spectrogram in video I use `dynaSpec package` in `R` that allows you to generate MP4 video of a spectrogram: <https://marce10.github.io/dynaSpec/>.

When I do a video recording of a vocalizing animal and I want to review both the visual behaviour and the spectrogram of the vocalization, I first create a scrolling spectrogram using `dynaSpec`. Then, I combine the video recording with the scrolling spectrogram using Davinci Resolve software (free version).

> **Comments:**
> **username_0**: This is a great answer to the second option, which is creating a scrolling video and adding it with video editing software. I'd prefer some software that just loaded the video and played a spectrogram in real-time. But your solution to the creation of scrolling spectrograms is a definite improvement over the way I create spectrogram movies. Right now I'm stuck screen recording. Now all I have to do is learn R.  
> **username_0**: I have a collaborator trying to make this work but he's running into difficulties. I'll report back if he gets it up and running. Meanwhile, I'm just screen recording Raven windows so I can move forward. That may end up being the easiest solution that is accessible to me after all.  
> **username_0**: My collaborator got the R package working, but it is painfully slow. He did a three-second file and it took 3 minutes. We set up an hour-long file and let it go overnight. It says it still has more than 7 days. Does this jive with anyone else's experience with this package? Or is something wrong on our side?  
> **username_3**: Hi @username_0 , I haven't used this package to date, but R can be notoriously slow with some (seemingly) straightforward processes. I'm assuming you're using a high performance computer to implement the package?  
> **username_2**: Hello, it is right that R is very slow to create such MP4 file. I think the package was created just for making a short video of scrolling spectrogram. Even with my gaming computer that is very powerfull, it takes ages to create scrolling spectro, while creating long videos with Davinci Resolve as example, is much faster.  

---

### Answer by username_4 on 2022-08-13. Score: 5

There is a similar question on [this Video SE](https://video.stackexchange.com/questions/23222/ffplay-is-there-a-way-to-display-a-video-and-spectrogram-in-the-same-output).
One suggestion/answer uses ffplay (linux?) to combine video with scrolling spectrogram.

> **Comments:**
> **username_0**: Thank you. I wish I could try that but I think this method is technically out of reach for me. No Linux knowledge.  

---

### Answer by username_5 on 2022-08-18. Score: 4

I’m not sure that this username_5 fully solve your problem, but for the alternative you have listed (generate scrolling spectrogram + overlay w/ video in adobe premiere), [ffmpeg](https://ffmpeg.org/download.html) is a useful tool to consider. The “[showspectrum](https://ffmpeg.org/ffmpeg-filters.html#showspectrum-1)” function from ffmpeg is a great option for producing scrolling spectrogram videos from a .wav by running a (relatively) simple shell script. This tool gives you lots of flexibility with spectrogram parameters, aesthetics, etc. ([a good example is found here](https://keunwoochoi.wordpress.com/2019/07/18/generate-a-spectrogram-scrolling-video-of-an-audio-file/)). You can create a similar .sh script for your needs and run from the command line to generate your scrolling spectrogram video (or many such videos) without screen recording.

This of course would still require syncing with video, but username_5 at least automate and greatly speed up your generation of scrolling spectrogram videos. As another commenter suggested, dynaSpec (R package) also appears to be a useful tool for this purpose but I do not personally have experience working with that package.

> **Comments:**
> **username_0**: Thank you. It's not clear to me how much time that would save me vs. screen recording, especially as I'd need to learn ffmpeg first. I think I'm just going to record the Raven window. It is realtime, but with my long recordings, I can just let it run. The screen recorder I have can just stop after a set period of time, so I don't have to stick around. I may look back at that option when I have more data, but for a few days worth of recording, I'll probably just go with what I know.  

---

### Answer by username_1 on 2022-08-17. Score: 3

I do not know how adaptable this set of tools is beyond the specific CATS animal-borne tagging application, but this paper provides a set of code/tools that pairs video, acoustic, and accelerometer data:

<https://animalbiotelemetry.biomedcentral.com/track/pdf/10.1186/s40317-021-00256-w.pdf>

The paper points to the GitHub repositories for the code and a workshop with tutorials and wikis to guide new users.

> **Comments:**
> **username_0**: Wow! That looks powerful and interesting. It could be overkill and would be a steep curve for me. Maybe for the next phase of my project where I hope to have more data and some little more technical support who could pull it together. Thanks.  

---

### Answer by username_6 on 2022-08-18. Score: 3

If you end up working with any of the tools in the GitHub repository mentioned by username_1 and run into any problems/questions, let me know and we can work through them together!

> **Comments:**
> **username_0**: Thanks so much for the offer. It looks like an amazing tool. I wonder if it is overkill given that I only have video and spectrograms. Also, if it would handle the 5.7 k video. Anyway, I think it is too much for me to pull off at this stage. It's a preliminary study with limited hours of data. I think I'm going to bite the bullet and screen record. Maybe in the next phase with more data.  

---

