Can Flash listen to a sound and analyze it?

[joshmehler]

Can Flash listen to a sound and then analyze it? For example, could it know the difference between 440 and 340?

Thank you,
Josh

[bit-101]

no

[joshmehler]

harsh

[bit-101]

hehe. sorry. there's just not much more to say about it.
check out swiftmp3

[Dickee]

FlashSoundAnalyzerClass () {
    var no,harsh;
    var soundFactor = boom;
    (no) ? hummm : hurtz;
    if (no.hurtz) {
        (harsh) ? hurtz : hummm;
        while (harsh.humm) {
            no.hurtz++;
        }
    } else if (no.humm) {
        (harsh) ? hummm : hurtz;
    }
    trace ('no: '+no%soundFactor+', harsh: '+harsh*(no%soundFactor)l 
}

// no: 320 hurtz, harsh: 400 hurtz;

Richard

[BlinkOk]

if flash can't analyze sound then how come you see sites with those equalizer displays (the bouncing graphs)?

[joshmehler]

I tried that FlashSoundAnalyzer code Dickee, but it doesn't work. I've tried changing come of the variables, you know, adding globals like, "bummer" and stuff, but it still won't go.

I'll keep messing with it.

Actually, that made me laugh out loud.

Thanks for that...

[bit-101]

Originally posted by BlinkOk
if flash can't analyze sound then how come you see sites with those equalizer displays (the bouncing graphs)?

99.9% of them are fake. just a looping animation that has nothing to do with the actual sound. as i said, check out swiftmp3. this can analyze an mp3 and send the data to flash separately to control a meter.

[brutfood]

I can only guess how mp3 coding works - but I imagine it may be possible to access some sort of short-term-predictor coefficients, or frequency information which could perhaps be transformed into suitable features for recognition.

In the old days, we used to use something called a Hidden Markov Model to deal with the variance in speed, etc, for pronouncing a word. I don't know if this technique is still popular.

However, If you're going to train, or even implement a recogniser, you may need something faster than ActionScript. -like a Digital Signal Processor.

Good Luck.

[fospher.com]

Originally posted by bit-101
99.9% of them are fake. just a looping animation that has nothing to do with the actual sound. as i said, check out swiftmp3. this can analyze an mp3 and send the data to flash separately to control a meter.

bah! thats a lie. a lot of them are flashtraks - check them out here at fk movies. so i believe in relation the question: yes, flash flash can recognize sounds

[bit-101]

no....flash cannot recognize sounds. if you can show me an actionscript command that can do so, i'd like to see it. flashtraks are encoded with a third party ap that analyzes the sound and creates a data.swf which contains hardcoded values relating to the sound. swiftmp3 is another third party ap that does the same thing. but i don't think flashtraks are used much at all outside of flashkit. swiftmp3 more so, but i still bet most sound meters are fake. they are very easy to create and really only serve as an indicator that the sound is playing or not. unless you are building a music site where the music is the main feature and you really want to highlite it with a real time meter. otherwise, it's just background music and noone really cares if the meter is accurately showing the sound wave.

[devnull_2k]

You got told fospher. hahaha

What were you thinking?

[fospher.com]

hehe, oh well, im not gonna argue. although lot of them are flashtraks, i've seen them used on many many sites..

cheerio!

[pitchdog]

I've seen flash look at volume using get mic.

http://www.artifactinteractive.com.au/mx/audio.html
http://www.artificialcolors.com/page_experiments_1.html

[bit-101]

yep, mx can analyze volume from the mike, but not frequency.
as far as flashtraks, you may be right. like i said, it would be warrented in a music oriented site where the song was the main content. i've just heard of swiftmp3 a lot more than flashtraks.

the main point was that it isn't flash that is analyzine the sound. it's a 3rd party ap doing that. that would be a nice feature for the next version of flash though. cross your fingers.

[pitchdog]

So ... I just got to thinking ... if there's a way for it to get volume w/ the mic, why not w/ internal sounds? That's not too much of a leap. I'm not even sure that's what you want, but I've played w/ the mic stuff and looked at the way differnt words or sounds appear. So in a sense flash listened to the sounds, and provided some data.

You could write some scripts to analyze that data.

(I don't have a clue how this works, by the way, its weird stuff and I just took a block of someone elses code to play with.)

[brutfood]

Originally posted by bit-101
yep, mx can analyze volume from the mike, but not frequency.
as far as flashtraks, you may be right. like i said, it would be warrented in a music oriented site where the song was the main content. i've just heard of swiftmp3 a lot more than flashtraks.

the main point was that it isn't flash that is analyzine the sound. it's a 3rd party ap doing that. that would be a nice feature for the next version of flash though. cross your fingers.

When actionscript looks at the microphone level: Are we talking about a measurement of power (squared values), or actual voltage level (which can be negative)?

DSP algorithms start with sampling the time waveform, although if what we're getting is a power measurement - this may complicate things. From a sampled time waveform, you can do an FFT to derive frequencies, or other methods: predictor coeficients, LSPs (Line Spectrum Pairs), or measure prediction gains across fixed filters, etc.

Although, I remember a guy at BT labs (where I used to work), did simple yes/no recognition in a completely time-based way. So it is possible for small sets of words that are very different in the way they sound (for all speakers/microphones).

There are two fundermental impediments with doing DSP processing in ActionScript.

1. Actionscript is very slow. Some would say very poorly engineered by Macromedia. DSP algorithms are notoriously compute-bound.

2. We usually need to sample at something like 8kHz. At a pinch, we might be able to go down to 4kHz, but flash can't even run at 4000 frames/second. So I think sampling real-time sound is out of the question.

So I think whatever we do, we're looking at files of sound. As I infered in my previous post, if we could access mp3 data (in flash, or back-end) then some of the heavy pre-processing is already done for us, and we may find this data contains some useful features to play with.

The question is: How much do you want to do it? This involves some research into the mp3 standard and file-format. I certainly don't have the time to look into this

PS: I'm probably alone with this opinion, but I don't want new features in the flash player. Only bug-fixes for the current ones, and much faster performance. Flash 6 introduced stuff that finally makes it useful, but I say stop there! New versions only split the users between those versions, and make it more likely that someone can't see your site. Besides the player is so poorly engineered that augmenting it with more poorly engineered features isn't good. I say, sort it out first.

[pitchdog]

I work w/ a guy who teaches kids w/ auditory proccessing problems. Anything that can help them "see" their speech is quite useful - even recognizing short vs long sounds.

But what else would you use mic volume for? I guess the question is -could you make a practical application with it, and if so, is flash the best tool? If not, what is MM doing with it? Oh, and there's the get camera also! I heard ideas that MM was working on conference tools via, Flash.

I'm not touching this, however, until I know what Macromedia plans to do with it. I'm aftraid these undocumented features won't stick around. Remeber the FS(Save)command.

>I'm probably alone with this opinion, but I don't want new features in the flash player.

Hmmmm ... I think I agree. I'd rather have a smoother Flash.

[devnull_2k]

Hm... personally I've always been a big supporter of the new thing. I love to have new toys to play with.

[Ian424]

about the camera etc stuff, they work with the flash comm server, i've seen a video conference in it work, it is pretty cool stuff, but as far as being really useful, i doubt it.