actionscript help for lipsync

[FLASHPULSE]

Ok, the next program I want to create is for lipsync but I need some AS help. Here's what I would like to do...

I would like to have a movie clip with multiple frames and be able to control it with AS, something like...

play frame 3

pause 3 seconds

play frame 5

pause 1 second

play frame 1

pause 7 seconds

As you see, I would like to jump from frame to frame at different times. Each frame having a mouth shape.

Frame 1.) "A,I"
Frame 2.) "E"
Frame 3.) "F, V"
Frame 4.) "L"
Frame 5.) "O"
Frame 6.) "U"

Frame 7.) "CL" (closed mouth)

Frame 8.) "C1" (custom 1)
Frame 9.) "C2" (custom 2)
Frame 10.) "C3" (custom 3)
Frame 11.) "C4" (custom 4)
Frame 12.) "C5" (custom 5)

The user will have to create his own mouth shapes in the 12 frames.

So when I create the program, you'll be able to sync a wav file to an animation then export the AS, so it can be used in KM. Would be great if the AS could be one script.

How does this sound? Any help would be great. :D

[FLASHPULSE]

It might be better to have a movieclip with 12 movieclips within it so there could be motion between each mouth movement.

[TDolce]

C'mon somebody help this guy!!!! KM really needs this!!!!

Todd

[necromanthus]

Set the FPS value to 30.
Now ... for each keyframe where do you need a pause,set this script:
del =30;
onEnterFrame = function(){
del -= 1;
if(del == 0){gotoAndPlay(x);}
}
stop();

So ...
for keyframe 3 you'll have: del = 90 , x = 5
for keyframe 5 you'll have: del = 30 , x = 1
and so on.

try to do your best
;)

[FLASHPULSE]

Thanks Remus, I'll start on this when I get home. :D

[FLASHPULSE]

Thanks again Remus but what I'm looking for is one script to go on the main movie clip that holds the 12 frames. So here's an example...

onClipEvent
goto frame 5
wait a second or two
goto frame 1
wait a second or two
goto frame 11
wait a second or two
and so on...

Does that make any sense?

[w.brants]

Hi Joe,

Can the code is posted a while ago be of some help ?
http://www.flashkit.com/board/showthread.php?&threadid=623645

Regards,
Wilbert

[FLASHPULSE]

Hello Wilbert, long time no talk. Thanks for the link, just have a couple questions.

mySoundEvents.push ( { sp:1500, o:_root.mc1, f:'gotoAndStop', p:['eeks'] } );

Looks like after 1.5 seconds something happens in mc1 that you goto and stop at. Not sure what eeks is for.

Does this script sync audio with movieclips and does the main script go into frame 1?

Thanks

[FLASHPULSE]

This AS doesn't work but shows what I'm trying to do in one script. I'm sure it also needs some stop commands as well.

onClipEvent(load){
counter = getTimer();
}
onClipEvent(enterFrame){

this.gotoAndPlay(frame 3);
}
if (getTimer() > counter+5000){

this.gotoAndPlay(frame 11);

counter = getTimer();
}

But I'll want to add as many "gotoAndPlay" frames and so forth.

[necromanthus]

Originally posted by FLASHPULSE
Thanks again Remus but what I'm looking for is one script to go on the main movie clip that holds the 12 frames. So here's an example...

onClipEvent
goto frame 5
wait a second or two
goto frame 1
wait a second or two
goto frame 11
wait a second or two
and so on...

Does that make any sense?
Ahaaa ... so you're looking for a sequencer. OK.
Create an empty movieclip in the first frame.
Add the following script for it:

onClipEvent(load){
faz = 1;
del = [0,30,90,60];
jmp = [0,5,1,11];
}
onClipEvent(enterFrame){
if( faz < 5){
if(del[faz]>0){
del[faz] -= 1;
}else{
_parent.gotoAndPlay(jmp[faz]);
faz += 1;
}
}
}


note: copy and paste this movieclip in each keyframe of your main movie.
That's all.
;)

[FLASHPULSE]

That's exactly what I'm looking for but can't seem to get it to work.

I created a movieclip and entered the script. Added my 12 frames in the movieclip and put text in each frame so I could see if the script works but I get nothing.

[FLASHPULSE]

Here's the file if you want to look it over, unless you're at work.

[necromanthus]

Originally posted by FLASHPULSE
Here's the file if you want to look it over, unless you're at work.
That's because the sequencer and the main movie are two different things (movieclips).
In your example is one and the same thing, and this is wrong !
Replace _parent.gotoAndPlay(jmp[faz]); with this.gotoAndPlay(jmp[faz]);.
Also, if you have a single keyframe animations, you have to add the stop(); action for each keyframe.
Come on JOE, you can do better than that !
;)

[FLASHPULSE]

LOL! I've been trying to work on this all night. Brain is sleepy at the moment. It's 2:44am here, going to bed and will work on it tomorrow, thanks Remus. ;)

[FLASHPULSE]

Ok, is this what you were talking about. The script seems to go through these frames in this order, 1,2,1,2,1,2,3, although that doesn't look like what it should do looking at the AS. Going to bed now.

[necromanthus]

Originally posted by FLASHPULSE
Ok, is this what you were talking about. The script seems to go through these frames in this order, 1,2,1,2,1,2,3, although that doesn't look like what it should do looking at the AS. Going to bed now.
Don't use tweens for AS keyframes !!!
How many times I have to say it ?

:D

[FLASHPULSE]

Hey Remus, after getting some sleep I got it to work both ways, with "_parent" and "this". The reason I was going with the "this" way, is so the main timeline could have all the animation (characters and so forth) and multiple movieclips could use your script for the lypsync.

I understand the way you were going about it with "_parent" now and that would work well also.

Thanks again! (just trying to stay in your black book, LOL) :D

ARRRG!!! This $%#@ server is always busy. Been trying to post for a half an hour now :mad:

[FLASHPULSE]

Hey Remus, can del have values of 1.25, 2.5 and so forth and play properly? The reason is, is because I need to get down to the milliseconds in my lipsync program. One second intervals is not quick enough for mouth movments. Also, in my program you can choose a framerate between the values of 12 and 30. I had to add that in because I knew if I didn't some one would complain that they couldn't choose their own fps.

Oh, the script is fine the way it is, no need to rewrite, just wondering if del will be able to use decimals? Thanks again. ;)

[w.brants]

As far as I can tell the routine Remus posted isn't accurate to the second but accurate to the frame. You can only display one mouth shape per frame so that should be enough.

But if you want it to be accurate I doubt if this approach will work. The only accurate approach for syncing is using sound.position because a flash movie will never play exactly at the specified framerate.

[FLASHPULSE]

Thanks for the information Wilbert. I knew that it wouldn't be acurate to the "T" or anything but I was wondering if I use a number that cotained a decimal if flash would accept it. In my test it worked but I wasn't sure if flash just saw it as (for example) 2 or 2.50? It was just a quick test to see if it would work. Not long enough of a test to see if timing went off down the line. That will be done later.

[FLASHPULSE]

edit message. redoing math :rolleyes:

[necromanthus]

25 FPS = 25 frames per second (40 ms per step)
30 FPS = 30 frames per second (33 ms per step)
and so on.
So ... you can't have 0.5 steps !
Also, flash can't be "accurate to the second" because setInterval doesn't work for intervals under 1/FPS.
For example,if the movie speed is 30 FPS, and you try setInterval(function,20) (in other words: 20 ms per interval - 50 FPS), in the runtime that function is called once per 33 miliseconds (can't work faster than 30 FPS).
As I said many times before (and as Wilbert told you),the unique solution in this case is to use sound.position
The reason: this command works asynchronous !
cheers

[FLASHPULSE]

Ok, for now anyway, my app will only use 15fps and 30fps until I can figure out this sound.position. It's almost complete and I don't feel like rewriting it now.

[FLASHPULSE]

You're going to kill me Remus. Please look at attachment.

[necromanthus]

Originally posted by FLASHPULSE
You're going to kill me Remus. Please look at attachment.
What else do you expect if your speed is 30 FPS and the del array is del = [0,1,1,1,1,1,1,1,1,...,1] ?!?
In other words: no delay at all !

p.s.
Works fine if you set the speed to 5-8 FPS.

[necromanthus]

I had to rebuild your "amazing" del array.
:xbones:

[FLASHPULSE]

I have to slow down my lipsync program from 1/30 of a second to maybe something like 1/3 and maybe that should do it. So when I export the script, del will be 10 when fps is 30. It's almost finished, thanks god. :D

[FLASHPULSE]

Here's a little test although I have to edit the script a bit before loading it into KM. Timing is somewhat off in flash.

http://www.flashpulse.com/koolmoves/lipsynctest.html

[necromanthus]

Originally posted by FLASHPULSE
http://www.flashpulse.com/koolmoves/lipsynctest.html
Well done !
But for future take care: in case of complex animations (large moving areas), the real movie speed is gonna be far away from the theoretical speed (you'll need a top machine to keep them closer enough).
To prevent all of these, your (future) sequencer has to be based on sound.position
;)

[blanius]

I had been thinking alone these lines too. Seems the best way would be to make a tool that plays the mp3 an you could press a key for different mouth positions as it goes and store it in an array that could then be imported into another flash that uses the same mp3 and this array file for the mouth positions.

[FLASHPULSE]

Originally posted by blanius
I had been thinking alone these lines too. Seems the best way would be to make a tool that plays the mp3 an you could press a key for different mouth positions as it goes and store it in an array that could then be imported into another flash that uses the same mp3 and this array file for the mouth positions.

That's what I've been working on. Everything works now, just cleaning some stuff up and writing a help file.
http://www.flashpulse.com/downloads/lipsync.jpg

[TDolce]

I'm excited!! This will be great!!! How much are you charging for this KoolMoves Plugin?


td

[w.brants]

Hi Joe,

It's looking nice. :)
What would really be great is if you could add a window that displays the wave file with markers on it's timeline. That way it would be very easy to see where the silences are.

[FLASHPULSE]

Originally posted by TDolce
I'm excited!! This will be great!!! How much are you charging for this KoolMoves Plugin?


td

After I get everything completed, it's going to Bob first. I've talked to him about maybe adding it to KM. This may or may not happen. After knowing what Bob's thoughts are and my own, I'll post more on this.

Originally posted by w.brants
Hi Joe,

It's looking nice.
What would really be great is if you could add a window that displays the wave file with markers on it's timeline. That way it would be very easy to see where the silences are.

Actually right now it's pretty fast to make a lipsync. I'm able to make a lipsync with a wav that's a few seconds long in about a minute (I'll explain more on how to do this quickly in the help file). So let's try it out after the release and if you still think it would be important, I'll look into adding it.

Also, I had reported in a post above about the timing being slightly off when used in flash. This was an error on my behalf on calculating time for the fps (which also now supports 12-30 fps and all seems pretty acurate so far).

I found one minor bug that I have to fix tonight but after that, I have to clean up my code and comment it (so I don't forget what's what) and make the help file. Won't be long. ;)

[necromanthus]

Originally posted by FLASHPULSE
That's what I've been working on. Everything works now, just cleaning some stuff up and writing a help file.
http://www.flashpulse.com/downloads/lipsync.jpg
Ha ha, I like it.
I'm waiting for more ...
And don't forget about that sound.position.
:mrpimp:

[tmoore935]

wow, this will make things too easy and put cartoonist out of work. looks like you have made everything real simple for anyone:D

[TDolce]

Originally posted by tmoore935
wow, this will make things too easy and put cartoonist out of work. looks like you have made everything real simple for anyone:D

Yeah,..speaking for just myself,...simple is fine with me. I'm probably the target when he aims for simple!!! :D

Hey FlashPulse,...you could even call it KoolTalk!!! (chuckle)

http://www.togotoons.com/KoolTalk.jpg

[FLASHPULSE]

Originally posted by TDolce
Yeah,..speaking for just myself,...simple is fine with me. I'm probably the target when he aims for simple!!! :D

Hey FlashPulse,...you could even call it KoolTalk!!! (chuckle)

http://www.togotoons.com/KoolTalk.jpg

LOL, that's a big play button. ;)

[TDolce]

Yes I love Big shiny buttons!!!:faded:

Always have, Always Will! Gives me that Mac feeling without a Mac.

So,..wassup with the Lipservice plugin?

Is this something that will be tied in with the next big release of KM? Have you already turned it into Bob for his decision?

Just curious and nosey! I'd like to know if I should start saving some pennies for the next release or just keep watching from the sidelines.

td:D

[FLASHPULSE]

Originally posted by TDolce
Yes I love Big shiny buttons!!!:faded:

Always have, Always Will! Gives me that Mac feeling without a Mac.

So,..wassup with the Lipservice plugin?

Is this something that will be tied in with the next big release of KM? Have you already turned it into Bob for his decision?

Just curious and nosey! I'd like to know if I should start saving some pennies for the next release or just keep watching from the sidelines.

td:D

I like fancy buttons myself. :D

Everything is complete except for the manual. I'll start working on it tonight. Then talk to Bob either thursday or friday to see what he wants to do or if I'll be offering alone, myself.

Here's a list of it's features:

1.) FPS can be from 12 to 30.
2.) Frames can be from 2 to what ever you desire.
3.) 10 mouth shapes (that can be combined for any word, I think)
4.) Save and open your work as a "*".lip file.
5.) Only imports standard "*"wav at the moment
6.) Delays for frames can be 1 to what ever you desire.
7.) A "Play" button to preview your lipsync animation before exporting.
8.) Of course, export your final lipsync animation to a "*".as file.

I've spent the last few days error checking everything to make it dummy proof, lol. It won't be long now. :D ;)

[FLASHPULSE]

Well, the lipsync program is complete, hooray!

I talked to Bob a couple of weeks ago and he sounded pretty busy, so I decided not to bother him with my program. That means I'm selling it myself. I'm trying to think of a good way to give KoolMoves users a discount? If anyone has any good ideas, let me know.

Hmmm... forum looks different.

[w.brants]

Hi Joe,

Nice to see you finished it :)
A few remarks.

I tried your KM example several times but the mouth movement starts before the sound file is loaded so it's out of sync from the beginning. I think you should have used the sound.position property. To support that theory I did a very simple fps test. On my computer when I specify a framerate of 30fps in KM, the actual rate it is played by the flash player is about 25fps. That would mean if you try to sync a streaming mp3 file of 30 seconds it would be about 5 seconds off when the sound stops.

Edit:
Sorry, should have read the help file before posting this. I see you were already aware of this point.

Second remark... it would be nice if one could choose from different sets of mouth images.

Good luck with your new application and keep up the good work,
Wilbert

[FLASHPULSE]

Thanks for trying it out Wilbert. :) Did clicking the refresh on your browser, fix the timing? Also, about adding a second set of mouth shapes, could you explain your reasoning on this?

[w.brants]

Did clicking the refresh on your browser, fix the timing? Also, about adding a second set of mouth shapes, could you explain your reasoning on this?

About the second set of mouth shapes...
I must have been not entirely awake :zzz: when I first tested your application. When I reviewed the demo I thought the mouth images were also exported but I see now you have to draw them yourself. Would be nice though if it would be possible to import some different sets of mouth shapes in KM. You did a great job drawing them but I doubt if I would be able to do it decently myself.

About the refresh...
I did click the refresh button but it made no difference. You are suggesting to use a preloader in you help file. It's probably the best option but would require to embed all sound files into the actual swf file making it huge if you want to do a voice conversation. Support for streaming audio would be great but would most likely require different scripting as I mentioned before. If you use external sound files and due to a dial up connection or huge server load the sound data is coming in to slow you would get unwanted pauses in the soundfile playback resulting in syncing problems.

A little idea...
Now you are using vector shapes. Adding support for sprites like Remus used in his mortal kombat game would open up the possibility to load a set of sprites inside the syncing application and use it for more general syncing things like for example a dancing figure. PureBasic would allow you easily to load sprite images inside your application. That way a user could preview immediatly the result with his own set of sprites.

[FLASHPULSE]

1.) I could draw some more mouth shapes for KM sometime, if this would help any. Then all you would have to do is scale them so they fit your characters size.

2.) In my own experience, having the audio files actually in the swf was best (with a preloader of coarse). Streaming would throw it off for dial-up users.

3.) New ideas are always great. I didn't go with the loading raster images for mouth shapes because this will create a bigger swf file but I might add this option later. Having the mouth shapes as vector images keeps the swf small and it can be used over and over with a different action-script. But of course, not everyone will see it my way, so new ideas are possibly new features.

[tmoore935]

I'd give a demo a try. If I can do something interesting, then anyone can.

[FLASHPULSE]

I'd give a demo a try. If I can do something interesting, then anyone can.

I'm sure you can tmoore! :)

I just email Bob again asking if he would like to offer tis as a plug-in and KM users would get a discount at about half off. He's a pretty busy guy though, so we'll see what his response is.

If you (tmoore) or Wilbert create a mouth animation with Flash LipSync, send me the *.fls file and I'll email you the action-script so you can get a feel for it. :)

[FLASHPULSE]

I just learned something. If you view a lipsync animation with just a swf file (without using IE browser) it plays fine. If you view your lipsync animation within IE browser it has to be 2 fps faster in the swf file. IE or browsers period must slow down flash a bit.

Here's a test and I had to set it 2fps faster.
http://www.flashpulse.com/koolmoves/example_lipsync.html

[necromanthus]

I just learned something. If you view a lipsync animation with just a swf file (without using IE browser) it plays fine. If you view your lipsync animation within IE browser it has to be 2 fps faster in the swf file. IE or browsers period must slow down flash a bit.
As I said a long time ago, any projector works 15-30% faster than its browser plugin.
The reason is simple enough: shorter way to system resources.
And as I told you before, I expect for "Flash LipSync v2.0" a sequencer based on sound.position
Also, Wilbert's idea (a kind of sprites library) is a great one.

keep up the good work
:cool:

[FLASHPULSE]

As I said a long time ago, any projector works 15-30% faster than its browser plugin.
The reason is simple enough: shorter way to system resources.
And as I told you before, I expect for "Flash LipSync v2.0" a sequencer based on sound.position
Also, Wilbert's idea (a kind of sprites library) is a great one.

keep up the good work
:cool:

I plan on working on both of those features in the near future.

Hey, I made it on the "Announcments" on FlashKits home page. :D

[Ultra Pulse]

Well this would make life easier, maybe there should be a feature where you import your own mouth shapes. Like for TH a certain mouth piece and so on. Did Bob ever see this topic?

[FLASHPULSE]

Well this would make life easier, maybe there should be a feature where you import your own mouth shapes. Like for TH a certain mouth piece and so on. Did Bob ever see this topic?

Curently the mouth shapes are set for Flash LipSync but when you use the exported actionscript in KoolMoves, you use your own mouth shapes.

If any KM user is interested in this, let me know, as KM users get a discount. I still have to set that up when purchasing.

[tmoore935]

http://www.diversioncentral.com/animate/flaspulse.html

This works in flash mx also

[FLASHPULSE]

http://www.diversioncentral.com/animate/flaspulse.html

This works in flash mx also

Well done, tmoore! :D

[FLASHPULSE]

Bob will be adding the following link to his support page. I thought I would add it here, so that way anyone who has read this and was thinking of purchasing it, gets the discount. 1/2 off for KM users. :D

http://www.flashpulse.com/koolmoves/flskmdiscount.html

[Ultra Pulse]

30 bucks, but it's worth it I'll see if I have any money left after this purchase. I downloaded the demo version, for some reasoon you made 'c' and 'k' look the same, or is that on purpose?

[FLASHPULSE]

30 bucks, but it's worth it I'll see if I have any money left after this purchase. I downloaded the demo version, for some reasoon you made 'c' and 'k' look the same, or is that on purpose?

If you buy KoolMoves, you get Flash lipsync for $15 USD. As for the mouth shapes, some shapes look the same because depending on how the letter is said in a word, some shapes look the same or about the same as others. There's actually only 10 shapes in the lipsync program. This keeps it simple and so when you create your own shapes in KoolMoves, you don't have to create alot of them.

Hope that helps.