Announcing the as3parser-framework opensource project
The as3parser-framework is a project culminating from over 5 years of actionscript3 lexing, scanning, tokenizing and parsing observations. This framework is fully independent of the Flex framework or any other library for that matter. A brief list of inclusions;
- AS3Scanner, AS3Tokenizer, AS3Parser (full AST including block ast)
- MXMLScanner, MXMLParser (full AST)
- ASDocScanner, ASDocParser (full AST)
- A Complete As3 DOM decorator framework wrapping the abstract parser node
- 100′s of unit tests covering all classes
Thanks to the PMD project in java, a rock solid as3 recursive decent parser was ported. All the other scanners and parser I created myself and tested0. The DOM framework comes mostly from the ASDoc documentor I created in java a couple years ago, that I still use when documenting all of my product components.
Here is the source code;
http://github.com/teotigraphix/as3parser-framework
All the code is released under the Apache 2 license.
How would you like to parse as3/mxml string data, or an .as file or an .mxml file from an AIR application or browser application? Well now you can with full AST.
The following is an example of the decorator node DOM.
var sourceFile:IAS3SourceFile = NodeFactory.instance. createSourceFile(source, "/home/src/my/domain/internal.as", "/home/src") as IAS3SourceFile; var compilationNode:ICompilationNode = sourceFile.buildAst(); trace(sourceFile.fileName); trace(sourceFile.compilationNode.packageNode.name); trace(sourceFile.compilationNode.packageNode.qualifiedName); trace(sourceFile.compilationNode.packageNode.typeNode.methods[0].name); trace(sourceFile.compilationNode.packageNode.typeNode.methods[0].parameters[0].name); trace(sourceFile.compilationNode.packageNode.typeNode.methods[0].comment.shortDescription); trace(sourceFile.compilationNode.packageNode.typeNode.methods[0].comment.getDocTags("see"));
I have a large Swiz project that will be using this framework, stay tuned.
Mike
For giggles, here is a class file it digests easily and converts into AST;
/*fpkg*/package foo.bar { import pkg.Stuff; import mx.core.mx_internal; use namespace mx_internal; /** * Class meta. */ [Event(name="myEvent",type="flash.events.Event")] /** * java doc */ public class /*f*/Blat extends Bing implements Febraz, Goo { private var data:String; private var data:* = womble(); /** asdoc for property */ public function get property():String{return null;} /** @private */ public function set property(value:String):void{} public function Blat() { super(); return; } [Annotate] [Basic()] /** * Meta doccomment. */ [String("foo")] [Num(1)] [Bool(false)] [Arg(foo="bar")] [List(foo="bar", that=2)] [Event("alpha")] [Event("beta")] private var x:String = 1; /** javadoc? */ public static function func(arg:Number, foo=null, ...rest):Boolean { default xml namespace = "http://example.com/"; for (var b=1;b<=10;b++) { bar(); } for (;;) { break; } for (var g in blah) { r(); } for each (var g in blah) { r(); } if (h==undefined) { throw new Error(); } else { /* bar */ } if (a) b(); while (false) v(); do { continue; } while (m); switch (blah) { case 3: x(); y(); case 4: next; default: x(); y(); } try { with (scope) foo(); } catch (e) { var a=b?(x+y):new Foo(); } catch (f:Error) { const X=new Thing(); } try { foo(); } finally { ff = function(y, z) { }; } l=[1,'2']; xx = doc.ns::name; m={a:null}; // TODO: hex = 0x3; d = <foo bar="{blat}"/>; d = <foo> <bar/> </foo>; r = /regexp/; a = b.(@c); zz = a..b; zzz = a..@*; zzzz = a..@["foo"+b]; /* unary expressions */ --a; ++a; a--; a++; a = -a; return a.call(i.j*k, l[m]); } } } // trailing comment /** * internal class. */ class InternalClass { private var foo:int = {}; public function InternalClass() { } } // trailing comment /** * internal function. */ function myFunc(arg:int=0):Vector.<int> { return null; }
Wow!
This is absolutely insane… Really well done!
I was just thinking, how difficult would it be to go the other way? In other words, from an AST to an ActionScript/MXML file? Effectively a file writer as opposed to a parser with this API?
Congrats on the great API!
Cheers
Hi Danny,
> Effectively a file writer as opposed to a parser with this API?
That’s already on the todo list. I developed the API with this exactly in mind. As of now, the API only has read, I wanted to get this stable on the read front(unit tests) before I started implementing the AST creation.
This is really alpha as far as the API, what will change is the write API on top. You will soon see createMethod() addMethod() etc.
Thanks for the compliment and check back. I’m getting a turnkey project going, once that is in place, I am integrating it into a Swiz workbench that will do what you just talked about.
Mike
Hey Michael
Wow, that’s perfect! Well, i’ll tell you what i’m interested in this for:
Robert Cesaric and I are working on an open-source Flash/Flex/PHP system called Aerial CMS which integrates Doctrine ORM & AMFPHP. The CMS does a lot of code generation and this parser would be PERFECT for creating/modifying generated classes/services/views/etc. I’m now following you on Twitter and eagerly awaiting that new feature
Cheers
@Michael Schmalle
Really cool…………… runtime coding………………..
http://www.thirstyneurons.com
Quite cool
Well hello there, just in time.
We’ll try to incorporate this into Moonshine IDE (written using AS3/AIR/Flex4/FTE).
http://thewaxup.com/erik/moonshine_showcase/
Feel free to help out if you want to. Might be too slow or lock for too long, we’ll see.
tata,
Erik
> slow or lockup
Well, in it’s very unoptimized state it parses UIComponent’s 13,000 lines with blocks in 2.8 seconds. (creating all ast and node decorator objects)
As a side note, this parser framework was really never intended for IDEs. If it works; fine but I wasn’t aiming for that direction.
This project was more directed at AS3 applications that create or manage code.
Mike
Very, very cool guys. I’m 100% certain us working on the Doomsday Console can learn tons from you. Regardless, i’m going to implement this as a DConsole2 plugin very soon.
Hacking! Awesome!
hi,I can’t get the source at “http://github.com/teotigraphix/as3parser-framework”,it just show 404 error page, how can i get the source code ? 3q
Th project is renamed.
In my GIT account it’s the as3-commons-asblocks project.
Mike