The fastest easiest way to get it right.

Design Ideas

When I sat down to write Andromeda, I had about 13 years of application experience behind me, and an absolute dread of spending the rest of my career doing what I had done before, no matter what level of management I had reached or might reach. I had come to the conclusion that most of the labor that goes into business database software was not only unnecessary, it was downright counter-productive. I needed to reconcile a growing desire to enter more customer-facing activities while being able to guarantee quality in a business that has no established methods for doing so.

Andromeda is my answer to those 13 years and my best hope for commoditizing the activity of business database programming. When I sat down to write it I did not believe it was all that unorthodox, but I have since come to believe it is quite different from any other framework or RAD tool. Beginning with some fairly respectable ideas about business rules, Andromeda freely takes what it likes and discards the rest from both OO and relational theory, ending in a toolset that eliminates 90% of the labor from the basic process of developing and deploying enterprise-class database applications.

Design Goals

These are the design goals of the Andromeda project:

  1. A Specification Must Be Accurate, Precise, and Comprehensible. All software projects suffer from the fact that programmers make mistakes implementing a specification that is itself a imperfect expression of needs that not even the customer clearly understands or has completely decided upon. Further, most programs, once written, cannot be adequately described by their programmers to the point of a foolproof reconciliation with the specification. Andromeda seeks to guarantee that we can express exactly what the specification is and that we can prove it has been implemented perfectly.

  2. Everything Routine Must be Automated. Most if not all frameworks will require repetitive actions at many stages of the development process. Andromeda aims to eliminate all repetitive tasks by automating all processes that do not require human judgment.

  3. The Solution Must Cover the Entire Application Cycle. Andromeda was intended from the first to be a tool that a developer would use during daily programming and that an end-user administrator would use to run an upgrade.

  4. The Assets of the Programmer Must Survive an Architecture Shift. Experience shows that when the business logic of a system is tied up in code, and the world moves on to a new language, the system dies, and somebody must invest in recoding it all. Andromeda shifts the programmer's assets out of code and into meta-data, which can be carried to the next programming language.

Ideas for The Specification

The Andromeda solution is, in short, to create a complete description of what must happen to the database, and then use a builder to build the database and library code to provide the run-time user interface. This basic approach happens to satisfy all four of the design goals stated above. The numbered ideas listed below explain how we determined what the specification would look like.

  1. All Business Rules Resolve To Database Specifications. While this may sound obvious and perhaps not worth mentioning, it must be stated explicitly to get the context right. Once upon a time our industry was called Data Processing (before IT, before IS, before MIS...), and the Andromeda mindset begins by restating this simple fact. An Andromeda project begins by working out what must go on in the database. The slick UI that gives the customer that "wow!" experience must be part of it, but it comes only after the database rules are worked out.

  2. People understand tables just fine. Most people have no trouble working with information organized into tables, even if they have never heard of terms like tuple and projection. Andromeda assumes that data will be stored in tables. Andromeda further assumes that you can freely discuss table design with customers, without creating some lingo to disguise this fact.

  3. We Don't Need and Can't Use Natural Language Specifications. The conventional definition of "Business Rules" includes a requirement that they be expressible in a human language. We reject that requirement. The vague and confusing nature of natural language makes it excellent for poetry and novels and terrible for specifications. Table specifications cannot be open to misinterpretation.

  4. We'll Call It Business Logic. To avoid confusion with the general meaning of Business Rules as requiring natural language, we use the term Business Logic when refering to our specifications.

  5. It's OK to use customer lingo. Given that we specify business rules in database terms, some customers will discuss "screens" very easily, while somebody who grew up with the web will say, "Give me a page where I can...." The Andromeda philosophy is that the customer knows their business and we know ours, and they don't need a lecture about proper terminology. If we know a screen means a table and a field means a column then that should be good enough to get the job done.

  6. The Irreducible Unit of a Database Application is a Column. The minimum requirement for a Business Logic language is the ability to specify precisely all possibilities for the definition of a column. The job of a programmer/analyst is to figure out what goes into the columns and make sure it gets there.

  7. Calculated Values Are First Class Citizens. To fully specify a database, you must include a variety of ways to generate calculated or derived values. Examples include copying a price from an items table to an orders table, multiplying that same price by quantity to get an extended amount, and summing those extended amounts to the order header.

  8. A Database Specification is Security, Constraints, and Automations. Building on the ideas above, a complete database specification, besides detailing such things as column widths and keys, will contain security, constraints, and automations. These three allow the specification of any and all business logic.

  9. A Specification Is Always Complete. Any particular application must have a single unified body of information that completely describes the database, there cannot be disparate disconnected sources. The programmer works on a text file, which is converted to tables in the database, and which is also converted to program files for the web framework. All three copies are different presentations of the same information.

  10. No Business Rules May Be Stored In Code. This is a natural consequence of the ideas above, but it is worth stating explicitly. Programming languages come and go much faster than the business rules that they implement, so it makes no sense to lock rules up in a format that will expire early. Further, code has none of the advantages listed here that meta-data has.

  11. The Specification File Should be Human Writable. Because the data file must be machine-readable, the ubiquitious and bletcherous XML format comes immediately to mind, but we don't use that format because human beings cannot easily type XML. As of this writing we use a home-grown CSS-like syntax, but will likely migrate to JSON or YAML as soon as time permits.

  12. The Specification File Must Be Machine Readable. If we are going to meet design requirement #2, that we automate as much as possible, then the specification must be readable by a machine so that it can be processed for things like database building and code generation.

  13. The Specification Is Itself Data (Meta-data). It so happens that a precise description of a database can itself be reduced to atomic data and stored into tables (this is called meta-data). Andromeda is fundamentally built upon this idea.

The Specification Itself

Considering all of the needs expressed above, we are ready to say at last what form the specification itself will take and how it will be used.

  1. Every Agent Gets Specification In Appropriate Form. In other words, we don't ask human beings to type XML files and we don't ask computers to read handwritten notes expressing incomplete ideas.

  2. The Programmer Works on a Text File. The spec is a simple text file that a programmer can work on in any text editor. This does not disallow a graphical design tool, so long as the tool can read and write the text file format.

  3. The Database Server Hosts a Tabular Version of the Spec. This is an absolute requirement if the system is to be self-describing.

  4. The Web Layer Hosts An Appropriate Version. Andromeda is implemented in PHP, which is very handy with associative arrays. Therefore Andromeda library expects to find the spec made available to it in PHP include files that contain associate array definitions.

The Advantages of Meta-Data

Knowing now that using Andromeda is about creating a spec and having the machine automate the process of using it, we can now spell out the advantages we gain from this process.

  1. Meta-Data is Platform Independent. The Andromeda developer's assets are his database specification files. While Andromeda at this writing builds databases on the PostgreSQL platform, the builder can be extended to generate SQL Server code, or DB/2 code. Building support for all Andromeda applications on a new server platform is an investment of 80-150 hours, rather than thousands of hours.

  2. Meta-Data is Architecture Independent. While slick user interfaces are crucial to ultimate success, experience shows they always change (and it always seems impossible to imagine what the next one will look like) and are replaced, while business rules tend to stay remarkably static. Andromeda gives the developer an asset that will survive major evolutions in IT architectures. As the browser evolves new abilities, and we see small-screens and heads-up displays, only small libraries of code need be refactored to bring to life the entire portfolio of Andromeda applications.

  3. Database Builds Can Be Automated. The spec goes into a builder program, which loads the specification into tables and generates DDL.

  4. A Specification Can Be Validated by Analysis. During the build process simple unique checks and referential checks can weed out such things as duplicate column definitions or bad data types. This is analogous to compiling program code and checking for syntax errors.

  5. An Upgrade is Atomic In Its Entirety. The build process examines the database structure, then looks at the spec, and generates a diff from which it builds a "plan", a list of changes that must be made to bring the database up-to-date. If any of the actions of the plan are illegal, such as converting a string to an int, then the plan is never executed. This prevents a production database from being stuck in a half-upgraded state.

  6. The Upgrade Plan Indemnifies Developers. In cases where there is extensive oversight of all database changes, the plan mentioned in the previous point can be printed for signatures before proceeding. (As of this writing this is not yet implemented as a feature).

  7. All Database Change Management Is a Single Process. There is no separate process for a build or an upgrade. In both cases the build examines the database and does what is required to bring it current. If it is empty, it is built from scratch, if it is already current nothing is done, and if it is out-of-date then tables are altered, added, etc.

  8. You Can Always Jump To The Last Version. Some database shops use scripts to handle database changes from version to version, requiring that every single script be executed in precise order to get from Version A to B. This is tedious to code, error-prone, and incredibly time-consuming at run-time. Andromeda has no such requirement. You can take a database at any version and bring it to any other version just by using the appropriate Database Specification File.

  9. Many Zero-Code Opportunities. Because the specification is a complete description of a database, we can write library routines that will read the spec and generate complete HTML for it. There is no reason in the normal case to do so much as create an empty class, the library routines can handle all regular cases with no further effort by the programmer.

  10. Human Judgment is Isolated to Beginning and End. An Andromeda developer puts all effort into two tasks, which are creating the database specification (in the beginning) and creating special presentation elements (at the end). This is one source of our claim to eliminate 90% of the time from the overall process, besides eliminating the simple typing of all of those table maintenance classes, Andromeda eliminates all of the need for human judgment during the coding.

Implementation Principles

So now we know that we will automatically build applications using a text file containing a complete database specification. At this point Andromeda parts company with nearly every framework out there, as we have no need for or motivation to adopt any kind of ORM or MVC strategy. Having a complete spec and automating the process gives us all we need, and the design now is about being careful not to let other ideas crowd in that will erase the benefits of the Andromeda approach. This provides us with our Implementation Principles, listed here.

  1. Use the Tiers As Designed. Databases are for data, the web is a go-between, and the browser is all about appearances. This means for instance that we have no need for ORM because we are not trying to reproduce relational databases in code or hide the relational nature from code.

  2. Cross-Tier Interfaces are First Class Citizens. This means that we are not seeking any strategy to hide the nature of the exchange between application code and a database server. The application sends SQL requests and receives SQL data, we are not trying to make it into something else or "abstract" it away.

  3. All Business Logic Is Enforced in the Server. At the end of a build, an Andromeda database enforces all of the security, constraints and automations defined in the spec. In casual conversation at SDS we say that an Andromeda database "can defend itself".

  4. Generated Server Triggers Contain Explicit Literals. All of the triggers on all of the tables are doing the same basic actions, such as retrieving a column from another table based on a foreign key. The various SQL statements are completely generated at build time, they are not generated at run-time.

  5. An Andromeda Database Requires No Special Interface. Andromeda enforces business logic with triggers, so that SQL can be used to access any table. This relieves application developers of the need to learn an artificial API of stored procedures. The placement of derived values into tables instead of views means no need to have both a view and a table.

  6. Users Connect With Real Accounts. Users of Andromeda systems use real database accounts. All of their privileges are determined entirely by their group membership, no application can add to or subtract from their usage privileges. This erases the worst effects of a successful SQL injection, as a user who cannot grant privileges will have no luck even if he can get a SQL injection through. This also renders less severe a bug in application code that attempts to show a user something they cannot see -- the user will see an error, but they won't see the data.

  7. Any Application Can Safely Connect to an Andromeda Database. Because the database can defend itself, and users connect with real accounts, any user of an Andromeda system can just as easily connect in with Excel and ODBC and they will be able to access exactly the same tables that they can access through the default web interface. There is no need to keep your customer's IT department out of the database.

  8. The Application is Self-Documenting. During the build, the complete specification is loaded to the database. The Andromeda framework provides that all applications can read the data dictionary tables and render them in HTML to give a complete view of the specification that was used to build the system.

  9. No Application Code is Generated During a Build. Some systems that aim for the same level of automation as Andromeda will have some type of build process that generates application code. Andromeda does not do that because the design of the application code is not coupled to the structure of the tables, it is likely to change far more often than the table structures and should not require a rebuild every time there is a change in the framework.

  10. A Build Never Overwrites Programmer Code. Some systems that generate code must go to great lengths to protect programmer files from system files. Andromeda does not need to do this because we do not generate files.

  11. Specification Is Converted To Appropriate Form. The build does not generate application code, but it does convert the specification into a form most easily handled by the application. In the case of PHP, this means include files that load associative arrays full of data about tables (which is faster than querying a database).

  12. Actual HTML Generation is a Two-Step Process. One of the tasks of the web layer is to generate table maintenance forms. This is a natural spot to use the data dictionary to generate textboxes, selects and so forth. To provide flexibility, the HTML is generated on the fly by library routines that use the data dictionary include files, and efficiency is gained by caching the generated HTML.

  13. The Web Layer Is Minimalist. Because the database implements all rules, the web server has no job except to turn HTTP requests into SQL Statements and to turn the results of SQL statements into HTML. The lack of business rules in the web layer eliminates the need for such things as MVC and ORM.

  14. The Programmer Must Be Able To Change Any Presentation Element. Some applications can run entirely on the framework's default table maintenance forms, while others need slick public-facing pages geared to specific tasks. Andromeda allows programmer override of the interface within a page, replacing entire pages, replacing entire templates, and providing multiple templates.

  15. Only Exceptions Require Code. Once a specification has been created, there should be no reason to create program files that can only duplicate the information in the spec. No programming is required for table maintenance forms that have no special UI requirements. This is one source of our claim to eliminate 90% of the time from the process, Andromeda does not seek to regulate and manage code, it seeks to eliminate it.

  16. The UI Is Built Along The Database Structure. Every table is made into a menu entry (tables are organized into modules for this purpose in the spec). A user can search, make new entries, edit and do other tasks according to their security privileges on that particular table. Foreign keys provide 'drill-down' links to child tables and "jump" links to parent tables.

  17. Everything Allowed To A User Is Visible. The web framework has only the goal of interfacing a user to the database. If they have privileges on a table then it appears on the menu and vice-versa. This greatly simplifies UI design, the code never has to work out complex resolutions of security privileges or risk a bug that presents something to the user they are not allowed to see.

The Bigger Picture

These three ideas close off what is required to make use of Andromeda in a development environment.

  1. Andromeda Is Radically Self-Implementing. A tool that can do what Andromeda does must itself be debugged, packaged, improved and otherwise developed. In other words, you need Andromeda to write Andromeda. Therefore Andromeda is self-implementing. Every Andromeda "node" has a program called a "Node Manager" which builds itself and the other applications on that computer.

  2. No Artificial Limits on Running Applications. Andromeda allows any number of instances of any number of programs to run on a server.

  3. Source And Version Control Must Be Included. The build process is so much a part of the process of developing and publishing Andromeda applications that it shapes the requirements of source and version control. Therefore Andromeda nodes must be able to exchange file updates, including patches to dev code or complete numbered releases.

comments powered by Disqus
Home |  Documentation |  Download |  Credits |  Contact |  Login
Andromeda © Copyright 2004-2013, Licensed under the GPL Version 2