Alright, a little background on what I'm making here.

This is (at some point, knock on wood) going to be a simple lighting engine. I have a basic polygon composed of points that a light source extends vectors from to the edge of the screen to calculate shadows.

Diagram for problem 1:



Here, point A is the light source and point B is the polygon corner casting a shadow. C is the point I would like to figure out (along the edge of the screen, bottom in this case). It's not hard finding the intersection of two lines, it's just messy figuring out which of the (in this case four) lines it intersects with first (top bottom left or right).

Diagram for problem 2:



Here the points B through F have all been cast onto the edges of the screen by the points along the blue line. In order to make a proper shadow I must connect the shape made by these points, plus the corner point A (here shown by the red and blue lines). For complex shapes which have concave features within concave features, the projection onto the edge of the window may not be in order, and may have a lot of overlap causing 'negative-fill' (when a single shape draws twists over itself flash considers it to be outside of the shape). Because of this, I need a quick, fast, easy way to sequence all of the points, OR just pick the outmost ones.