Sunday, July 31, 2005

 

More Dead Air, and More to Come

All right, folks; sorry again for the huge delay in posting. I've been letting my subconscious turn over the design (read: slacking off), and there are a few changes that will need to be made to some of the interfaces that I've defined so far. I'd really like to get at least the SWFIO design done before the end of the summer, but I don't think that's going to happen. Which leads to my next announcement…

I'm going in tomorrow (Monday) to negotiate a short-term, full-time job at a software company I have prior dealings with. Short of dire budget problems or a major catastrophe, I suspect I'll be employed starting August 11th to September 27th, and school will be starting up directly afterward. Furthermore, I'm going to be on a road trip and unavailable from August 4th to the 10th. Thus, I'm probably not going to have too much time to do anything really substantial for the rest of the summer.

Wednesday, July 27, 2005

 

SWFIO Filters Design

All right, I sat down and hammered out the basics of the SWFIO Filters package. I still haven't documented the semantics of all of the calls yet, but I know what they are (I just need to write them down). Once I get the semantics documented, I can write the unit tests — after that, I can implement the code, or leave that for someone else to do while I move on to designing other pieces.

I've updated the Umbrello diagram on the server (check the links at the left), and have also generated an image of the diagram (click the image below to see the full-scale picture). Enjoy :).

An unreadable, low-res UML diagram representing the SWFIO filters package

Monday, July 18, 2005

 

Cairo and GPLFlash

All right; I'm going to really drive the "Cairo will/won't work for us" argument into the dirt. I think it will work for us, and I'm prepared to do the research to back that hunch up.

This email is a bit deceptive on a couple of points, though not intentionally so. Cairo, first off, is a vector drawing library, which means it talks in terms of paths (curves/lines), strokes, fills, and transforms. Both SWF and SVG are vector-based formats; SWF uses only paths, strokes, fills, and transforms, whereas SVG uses these in addition to primitive shapes.

The big issue for correctly rendering SWF shapes is how fills are performed. The aforementioned email states that there are "left/right area[s]"; drawing on the spec, I'll elaborate a little. The specification says that there are two fill types that a path can have: "fillstyle0", which is applied to the left of the vectors in the path, and "fillstyle1", which is applied to the right of the vectors in the path. However, due to the examples and language used after this statement, I am under the impression that "fillstyle0" is for interior fills, and "fillstyle1" is for overlap/self-intersection fills, and that the left/right nomenclature was purely for the sake of example. The only way to find out will be to run a test through MM's player; I'll try to construct such a test in the near future.

Update: I have constructed tests. I have been totally unable to get MM's player to render "fillstyle1" and "fillstyle0" simultaneously. MM's player, for all intents and purposes, behaves exactly like "even/odd" filling, which Cairo provides. If anyone is interested in getting the test files and source code used to generate those tests files, ask me for a URL in IRC (any link posted here would probably go stale too fast).

Here's a table that compares SWF's functional needs with the drawing tools that Cairo provides:

SWF-requiredCairo
RGB/A Solid fillProvided (sample)
Linear GradientProvided (sample)
Radial GradientProvided (sample)
Clipped Bitmap FillNot provided
Tiled Bitmap FillProvided (sample)
Stroke widthProvided (sample)
Stroke colorProvided (sample)
Moving Without DrawingProvided (sample)
Affine TransformsProvided (sample)
Drawing LinesProvided (sample)
Drawing Curves*Provided (sample)
Even/Odd FillingProvided (sample)

*Note that the Bezier curves must be converted from quadratic (SWF) to cubic (the rest of the world :p). This is a straightforward transformation.

The only thing I can see standing in the way of using Cairo is the "clipped bitmap fill," which takes a 1 pixel outline of the bitmap, then extends that outline as far as it needs to go to fill the area. I can't easily generate a test for this functionality to see how it works in MM's player, so I don't know if it will be easy to emulate (using something like imagemagick) or not.

Sunday, July 17, 2005

 

Overdue Update

Sorry about all of the dead air lately. I have been rather distracted these last few days, so I don't have much to show :p. There are still a couple of news-worthy tidbits, despite my lack of concentration.

ffmpeg Dependency Issues

First off, thanks to willem for helping out with this matter.

So, it turns out that ffmpeg actually generates .pc files (for the pkg-config system), but how those files are handled varies from distribution to distribution. Under Debian, an ffmpeg-config script is installed, and other under distributions, the .pc files could be installed. Under Gentoo, none of those files is installed. Thus, I've re-written our ffmpeg test so that it looks for "ffmpeg-config" first, "pkg-config avcodec avformat" next, and then finally tries to just right-out look for "-lavcodec -lavformat" (or "-lavcodec_pic -lavformat_pic").

The main reason we have to use ffmpeg-config or pkg-config first is because on systems like Debian, the ffmpeg libraries are cross-linked to lots of other optional libraries. Thus, there is a variable list of -l flags that must be used in addition to "-lavcodec -lavformat", and it would be foolish to try to brute-force all of the possible combinations.

Edits & Corrections

kermit_ has kindly donated his time and skills as a technical writer. He's already made a few adjustments to the GPLFlash homepage, and he's presently working on annotating and cleaning up the contents of gplflash2/doc/. Thanks kermit_!

NPAPI updates

I've made various updates to the NPAPI docs (html, pdf, and texinfo source). The initial documents were totally, flat-out wrong in some respects. I could really use a Windows programmer and a Mac programmer to help me write the platform-specific parts of the guide. Anyone interested?

Tuesday, July 12, 2005

 

Scouting Documentation for NPAPI

I'm writing something that I've dubbed a "scouting document" — documentation that covers a whole topic that will affect us in the future, but only a subset of which I'm interested in at the moment. In this case, I've decided to take a detour and start working on a thorough and comprehensive NPAPI guide (since even the official documentation has some holes and misinformation). When someone in the future goes out to re-write our plugin (as I'm sure will happen eventually), this document will hopefully be the only thing they need to understand the requirements on the plugin side of things. I've been working on it for about two days now, and hope to have my research and first draft completed within a week. For those interested in alternate formats, the texinfo source document and pdf rendering are also available.

Monday, July 11, 2005

 

Explaining SWFContext

Someone, whilst looking over the diagrams in a previous post, asked me why the SWFContext package depended on the SWFIO package. The answer to that initial question is because the SWFContext package will (or has the high potential to) handle SWF tags, structures and/or primitives. I think it would be highly unlikely that, throughout the entire package, not one class or sub-package would handle objects from SWFIO. Handling (sans a higher-layer abstraction or wrapper) is enough to cause dependency.

The path that our discussion took, however, went towards "ActionScript only" coding, and how it wouldn't make sense to have just tags for keeping context in that case. This is entirely true: in a file where you have only two tags (an ActionScript and an End), the vast majority of the work is not going to be done with tags; it'll be done in the ActionScript VM, using context-saving methods that are designed for that purpose. However, note that there is no conflict here: SWFContext is a package, not a class. I used packages on purpose, because I really don't know enough about how the system should work overall (yet) to create classes.

Now, I surely haven't thought deeply about AS and how to design its implementation yet — and I most certainly didn't think about the situation of a completely programmatic SWF. However, the design does allow for these "future requirements" by creating appropriate extensibility points. To implement AS, we shouldn't need to change anything in the SWFContext package at all, nor should we have to change anything in the TagLogic package (aside from implementing the AS tag handler). Creating things inside of AS will be managed by the AS VM; the context will be allocated by that VM, and could then be stored in a variety of places (such as a void* in an STL map container, either in the TagData object itself, or back in some context object in SWFContext).

Now keep in mind that I made up most of the last paragraph you just read (that is, I hadn't planned it out in advance). The design is far from being coherent, let alone complete — it's just a rough roadmap for the time being. In fact, a very important detail (having a SWF- and/or tag-central location where multiple "plugins" can store independent contexts) came out of the exercise. I'm nowhere near touching AS, or rendering, or user input, or sound, or A/V sync, or any other of a million other broad and nuanced areas that the final player will encompass. That's why getting questioned and challenged along the way is ultimately a good thing — other folks can see the potential mistakes I'm making, and that gives me the chance to plan out and ensure I don't make that mistake. That is, after all, what design is about ;).

 

Asking the Mozilla Devs

All right; the long and short of it is that the "simple" plugin from last time didn't register or otherwise work. As such, I decided to pop over to irc.mozilla.org and see if I couldn't straighten things out directly with the developers. Apparently, the XPCOM plugin interfaces are deprecated >_<. So… there's no reason not to just use the plain-jane C NPAPI interface. I'll go ahead and design with that in mind.

Sunday, July 10, 2005

 

Pulling Mozilla CVS

So, after looking at several "real-world" examples, it turns out that all of them (including the mplayerplug-in project) use NPAPI in order to actually register and communicate with the browser. There are, however, sample plug-ins in the Mozilla repository that appear to not use the old NPAPI at all. I've been examining these plugins, but have not had any luck getting my own version to show up in about:plugins. I'm currently building the whole Mozilla suite from CVS now in order to see if the sample plugins actually register properly, or if it's just me doing something wrong in my implementations.

For the curious, here is the link to the "simple" sample plugin that I'm trying to build. I think I've learned a lot about the architecture thus far, but since I have yet to get a plugin working, I can't be sure. As such, I'm going to refrain from documenting my "findings" from studying that plugin until I'm sure that what I think I've learned is fact ;)

I'm also starting to wonder if this whole process is worth it. Since so many other plugins use the old NPAPI, why not go ahead and use it for GPLFlash? The old API certainly seems like it would be easier to use. I guess the big thing is the scripting interface that can be exposed with the new API, though that can be done in conjunction with using the old API calls for plugin registration. Ah well — the "simple" plugin sample just finished compiling (with a little bit of tweaking), so I'm off to test it.

Saturday, July 09, 2005

 

Pulling Real-World Examples

Well, I decided I'd go see how other folks have implemented plug-ins, and use that as a basis for building my own. I started out with the plug-ins that I have installed: gxine and librsvg. Both of these use the "old" NAPI style. Then it occurred to me that someone had mentioned another plug-in that we should borrow from — the mplayer plug-in. I fetched the source for that plug-in again, and lo and behold — it uses XPCOM. I went back and grep'd my logs to see who suggested that; thanks tgc! You had the right idea, you were just ahead of me in the scheme of things. I'm going to spend today alternating between doing housework, and picking apart this plug-in.

 

XPCOM and Gecko Plug-Ins: Help?

So, it looks like the interface I currently have a link for (on in my "reference" links on the left) is for the "pre-5.0" plug-in system. Apparently, there's a new system based on "cross-platform component object modules" (XPCOM). This means that it's OO, interface-based, and roundabout ;). Unfortunately, there doesn't seem to be very good documentation on how to use the XPCOM system to make a plug-in, so I'm a bit stuck. I was preparing to write a little program to figure out how Mozilla/Firefox's plug-in system works, so that I could better design the SWFIO::Filters::NAPI::Read class.

Does anyone have familiarity with the XPCOM model in Mozilla? Does anyone know of a good reference for writing a plug-in based on this model? If not, I'll do my best to document my own path.

On an unrelated note, I'm going to try and be more regular with my updates. It appears as though folks are actually reading this thing now, and it's rude of me to be too far off in my corner without updating you all on what I'm doing. I'll do my best not to dilute the posts, but I'll also be more forthcoming with the snags I've hit, and the intermediate steps I'm going through.

that's all for now :)

Friday, July 08, 2005

 

Anon. Comments

This is a bit of a delayed announcement, but I have turned anonymous comments on in order to allow a wider range of people to respond to the posts.

Wednesday, July 06, 2005

 

A Walk Through Design Lane

All right, I finished documenting my topmost level of design, as well as the first (of many) second-level design diagram. I'll be presenting these in this post; the original information is all available in the current design document.

Top-level design

four packages: SWFIO, SWFContext, Frontend and TagLogic. SWFContext depends on SWFIO, and TagLogic depends on all of the other packages.

Because of the dependencies, the packages should be implemented in this order:

  1. SWFIO
  2. SWFContext
  3. Frontend (can be done concurrently with 1 and 2)
  4. TagLogic

Package Descriptions

SWFIO

A strictly data-oriented interface to the SWF format.

Handles reading and writing all primitive data types, and data-only structures from an SWF stream.

SWFContext

Provides different levels of containers that, as a whole, represent the state of an entire SWF file.

The SWFContext package is intended to provide a common access point for external SWF processors (such as SWF interpreters, writers, or even dynamic filters). Some of these applications will want the SWF container to be smart (in the case of writers), whereas others will want the container to be as dumb as possible (in the case of interpreters and some special filters). This is why SWFContext is a package, and not just a single class.

Frontend

A normalized interface to shuffle data to and from the user.

Any implementation of the Frontend package will need to provide functionality for reading keyboard events, reading mouse events, displaying graphical objects, playing sounds, and possibly reading graphical objects (webcam) and reading sounds (microphone).

TagLogic

A behavior-oriented interface to the SWF file format.

TagLogic requires SWFIO for the TagData package it contains. Each TagLogic class representing a flash tag has a corresponding SWFIO TagData class embedded within it. TagLogic also requires the SWFContext package, since tags may need to refer to and modify other tags in the SWF file. Finally, TagLogic also requires an appropriate Frontend package (which could be implemented multiple times, allowing for the Frontend to be switched) in order to get input from the user, and provide the appropriate visual/audio output.

SWFIO package

Four packages: Filters, Datatypes, Structures, and TagData. Datatypes, Structures and TagData depend on Filters; Structures and TagData depend on Datatypes; and TagData depends on Structures.

Because of the dependencies, the packages should be implemented in this order:

  1. Filters
  2. Datatypes
  3. Structures
  4. TagData

Package Descriptions

Filters

Provides a framework for normalized I/O and drop-in data filters.

This package defines filter classes that can be "chained" together, similar to a pipe on the command line. Filters can perform an in-line function (like encryption/decryption, compression/decompression, executing a SED script, etc.), normalize access to an existing data source/sink (STL iostreams, C file I/O, NAPI asynchronous I/O, etc.), and/or provide "special" data extraction/insertion methods (SWF primitive data types).

Datatypes

Provides a way to access SWF primitives that are not directly provided by the SWF filter.

Some primitive data types (most noticeably the fixed-point types) are directly derived from other SWF primitives. As such, these types are not really primitive, but alternate interpretations of "true" primitive types. This package sits on top of the SWF filter, and provides the primitives that the SWF filter does not directly support.

Structures

Contains classes for data structures that are used by several different SWF tags.

The structure objects in Structures do not have any behavior other than asserting that the structure values are always within their bounds and providing correct serialization/deserialization to/from an SWF filter.

TagData

Contains classes that read and write entire SWF tag objects.

The tag objects in TagData do not have any behavior other than asserting that the tag values are always within their bounds and providing correct serialization/deserialization to/from an SWF filter.

 

I'm Back!

All right; now that I'm back from my hiatus and settled in, I'm ready to go back to doing some serious work. Here's what I'm planning to take care of, in order of importance:

  1. Getting more work done on the new design.
  2. Assisting newbies on IRC and the mailing list
  3. Helping the wiki move along.
  4. Reviewing the "detailed" design document for the current system.
  5. Replying to E-mails. My apologies to folks on this one, but crafting an E-mail is a 30 minute to 2 hour job for me, and I can't really work on anything else in that period. Catch me on IRC!

Expect updates on the design soonish. I'll try to generate some png files for important parts of the system for those who can't or don't want to muck with Umbrello.

Note: I have been told by someone who recently downloaded the SWF specification that Macromedia has added a restrictive license to the end of the document. Because of this, newcomers will not be able to get a copy of the spec and still be able to contribute to GPLFlash's development. Since other developers (such as myself) already have access to the specification without the license, this should not pose a major problem in the short term. Please bear with us, and do not seek out or use this specification in conjunction with the GPLFlash project.