MVP
07 Apr 2009

This is the start of a story about a three-year companywide Software Factory effort. The result is not your common code-generating or process-focused factory. In our effort to realize maximum value, we focused on solving common challenges that existing factories do not address.

We created a Factory to automate and standardize development and deployment on top of Microsoft Server products, targeting shared infrastructures with .NET, SQL Server, SQL BI, BizTalk and SharePoint:

Interested? Read on…

History

For the past three years it has been my job to move Macaw, a Microsoft Systems Integrator, to a software factory approach.

Macaw specializes in realizing IT solutions with the latest Microsoft technologies. While Macaw is organized in several Solution Centers that each specialize in specific types of IT Solutions, it is common for Macaw’s clients to run these different solution types on a shared infrastructure (deployed on the same set of machines) and a common Microsoft technology stack (deployed on the same Microsoft server product installations, e.g. SharePoint). We created a Factory to support this.

Three years ago, based on experience gained at Macaw’s (then) largest customer, I proposed a vision to create a single, durable software factory to support 80% of Macaw’s work as a Systems Integrator. The vision was recognized and adopted by Maarten Sikkema, co-founder and CTO of Macaw, and with his support things were set in motion. I became member of a dedicated Factory core team, which was tasked with designing, building and embedding the Macaw Solutions Factory company-wide.

I convinced Macaw’s key technology experts and development community leaders to participate in creating the core Factory standard – the Microsoft .NET Architecture Style (MAST). The entire Factory initiative is driven by voluntary participation (building the Factory) and adoption (using the Factory), as opposed to a mandatory approach. Although this did take a lot of extra effort and discussion to get people on-board, the resulting commitment was strong and durable because people really believed in the initiative and they felt it was also their factory. Durable community commitment is key for a durable factory approach; this is a necessary and worthwhile effort.

After the first year, Serge van den Oever joined the core Factory team. As Macaw’s resident SharePoint guru he started out adding SharePoint support to the Factory, but since then Serge continued to add many key innovative elements to the Factory (PowerShell integration, the Factory Wiki, and the Factory Guide Visual Studio extension).

As Factory usage became more and more widespread, an extended team of Factory Champions was created, with members from all solution centers. The Factory Champions were tasked with continually improving Factory usage and usability for their respective centers. Finally a strategy was put into place to integrate results of Macaw innovation projects into the Factory.

During this time, we consciously maintained an internal focus, supporting and strengthening Macaw’s capabilities as a Microsoft Systems Integrator.

Today

The Factory has been adopted throughout the company and has been sold under license for use by non-Macaw personnel. The people and processes to continually improve the Factory are in-place. We are integrating the latest Microsoft technologies as they are applied and we capture best practices as we learn them, across all solution centers. Now we feel it is time to share what we have learned and achieved.

In several factory blog post series, Maarten, Serge and I will cover the Factory vision, overview, features, usage and customer cases.
You can use this feed to easily track Factory posts by all contributors.

Vision

Ok, enough history. Since this is the first installment in the Factory Vision series, let me outline the key elements of our vision on a Software Factory for a Microsoft Systems Integrator.

Note: for this introductory post I will stick to the high-level view; it is long enough as it is. However we intend to quickly zoom in on the key details in the next posts, right down to the ‘bare metal’.

Design goals

When designing the Factory, we set these goals:

  • Fit at least 80% of Macaw’s work as a systems integrator, specialized in the latest Microsoft technologies.
     
  • Be effective for very small teams (1 – 2 persons) as well as large teams (15 persons).
     
  • Support multiple, independently versioned solutions running on a shared infrastructure. Large projects or programs are often organized into multiple teams of 5-8 persons each, where all teams work on the same chain of DTAP (development, test, acceptation, production) environments. This means that the Factory must be versionable per solution; different factory versions must be able to live side-by-side in a shared development environment.
     
  • Be effective for short, simple projects (a simple web site) as well as long, complex ones (a SOA).
     
  • Deliver “Tomorrow’s IT, Today”,
    company-wide. Allow the knowledge and experience of Macaw’s top
    innovators who are working on projects to be captured with minimal overhead, and then distributed company-wide, quickly. This requires easy factory customizability in an ongoing project, frequent factory releases, and easy factory upgrades (also in ongoing projects).
     
  • Be durable, meaning that the effort in maintaining the factory – integrating new technologies and practices as they are learned by Macaw – should remain constant, while the total generated value of the factory keeps increasing. This means embedding Factory maintenance and innovation in the organization, and partitioning the Factory components across the underlying Microsoft server products and across the different versions of those server technologies.

Overall vision

Overall we envisioned a durable, company-wide Factory approach that would relieve people from the same tedious, error-prone, repetitive tasks that they encounter on many projects. Instead, they are able to focus on what is specific and innovative for the current project, while the Factory provides them up-to-date, combined knowledge and experience of their best colleagues at their fingertips. This approach will bring every team closer to the level of the company’s very best combined; a dream team.

Core Factory value

Key to the value of the Factory is the value of Macaw’s best innovators, who are leaders in their respective fields, working in innovative projects. The Factory amplifies this value (making it available for other teams) and also makes it more durable (even after the expert leaves the company, the captured knowledge and experience remains available). In a market where top IT talent remains scarce, hard to come by and hard to retain, this Factory approach provides a significant strategic advantage for an innovative systems integrator.

Customers benefit from consistent high levels of productivity and expertise that – across the board – are simply unattainable for traditional systems integrators who still use a manual, one-off approach to development. When customers license the Factory for use by non-Macaw personnel, they boost those developers with Macaw’s packaged expertise.

What can you build with the Factory?

The first step in defining a Factory is describing what types of solutions you can build with it – what the targeted software product family is. We observed a large overlap across different solution types with respect to what Microsoft server products are used and what best practices are applied. In addition to that, different solution types often coexist on a single shared infrastructure.

So instead of building a separate factory for each type of IT solution, we decided to define a single Factory that can build:

“Solutions composed from a combination of custom .NET code and zero or more Microsoft server products”.

The phrase “zero or more” indicates that we target solutions ranging from the ‘traditional’ fully custom solutions to solutions based on standard server products. The latter type of solutions has become predominant for Macaw’s clients; as much as possible of the functionality is bought with standard server products. The remaining functionality is then realized by customizing and extending the standard products:

  • Examples of custom components are web pages, console UI or windows UI, business components, workflows and business entities.
     
  • Product extensions are specific types of components that Microsoft has defined to allow extending the functionality of their server products through code, such as web parts for SharePoint or pipeline components for BizTalk.
     
  • Product configurations are product-specific settings and data that specify how to use the product’s existing functionality, such as configuration of web sites and web applications in IIS, site definitions for Moss 2007 and custom databases for SQL Server.
     

The Factory supports building small or simple solutions equally well as building large or complex solutions. The Factory is automatically configured to only include the tools and templates that you actually need for the solution at hand. If the solution is ‘light’, the factory will also be ‘light’, e.g. to build:

What does the Factory do for you?

Like most factories, this Factory increases developer productivity and quality by automating repetitive, error-prone development tasks. When building the above types of solutions, this is how the Factory addresses the most common big challenges for you:

Challenge 

Strategy

Install development machines
For every person working in a development environment, a project-specific development machine must be installed with all required server products in a consistent configuration. This often involves many manual, error-prone steps; it is quite common to spend several working days installing and configuring a development machine correctly. Inconsistent installations also cost time downstream due to incomplete functionality, local defects or inability to reproduce defects from other environments.

Automated Factory Machine Installer
Virtual development and test machines are created fully automated, providing multiple OS choices in 32 and 64 bits variations. From a single selection menu all required server products are installed fully automatic on each machine in a single scripted step, with all the latest service packs and patches.

Deployment

Deployment needs to be done many times during the entire application lifecycle, throughout different environments. However extensions and configurations for Microsoft Server products require a wide variety of deployment tools and many error-prone, manual steps.

Automated deployment
Fully automated deployment across all DTAP (Development, Test, Application and Production) environments.


Maintainability
Maintaining a clear, consistent, stable source structure over time and across solutions for a wide variety of component types while developers, functionality and technology change.

Standard implementation structure
The Microsoft .NET Architecture Style (MAST) standard provides a standards-based, durable, stable yet flexible implementation structure that is consistent across Microsoft server products and technologies. MAST offers separation of concerns, alignment with underlying server products, and cost-effective automation through convention over configuration.

Specialist knowledge
How to make all required specialist knowledge of the underlying server products available in each development team, while specialist fields are narrow and scarce specialists are distributed across the organization.

Community driven guidance in context
Like most factories, specialist knowledge and best practices are packaged in templates and tools.

Going beyond that static approach, we also provide community driven guidance in context. Within Visual Studio the actions that are relevant in the current context are displayed. Guidance on each action is available in a single central Wiki – both approved guidance and community content. When any community member enters some guidance, it instantaneously becomes visible and available for all Factory users, when they are working in a relevant context.

 

New world of work
We provide the infrastructure to enable all employees to work together from any location, ad-hoc. They can work together on the same machine screen while talking to each other, allowing them to work together as least as effective as when they were physically working on the same machine.

 

What’s in the box?

So now we have seen what types of solutions you can build with the Factory, and which common challenges the Factory helps you address. Now it’s time to look at what exactly the Factory itself is. Here is our governing definition of the Factory:

As you can see this is quite a broad definition for a factory. We explicitly include not only the development tools and templates, but also all infrastructure that enables the Factory users to work (together) from any location. One of the main pillars of our factory is to support the “New World of Work”. This is part of our strategy to handle the challenge of distributed specialist knowledge.

How do we ensure sustained, increasing Factory value?

From the outset durability was a design goal for our entire factory approach. The main steps we took to realize this were:

  • We embedded Factory usage and usability in the organization by creating the Factory Champions team and working closely together, while keeping Factory usage a choice.
     
  • We proposed the Macaw Innovation Board, as a central place to provide overview and direction on all innovation projects. This allows to integrate and align innovation projects and the Factory to each other.
     
  • We created a Factory Improvement Forum with voting, open to all Factory users, to allow community-driven feature proposal and prioritization. This ensures that we are always working on the right features, that are the most wanted and valuable for the project teams.
     
  • We partitioned the Factory components across the underlying Microsoft server products and across the different versions of those server technologies, so we can distribute our maintenance effort proportionally to how much value is currently generated with a specific technology. Obsolete (versions of) technology can be removed in newer Factory versions very easily, while older Factory versions that still use that technology can continue to live side-by-side with newer Factory versions.
     
  • We made the Factory implementation very open and easily modified and extended. We implemented a large part of the Factory functionality in open PowerShell scripts to allow quick customization without the need for separate Factory source code or a Factory development environment. Our solution templates are customized with the same ease as when coding in regular Visual Studio Solutions. This way the innovation can be driven by projects on the go, modifying their factory instance as they learn.
     
  • We provide on-call consultancy from the core Factory team to help innovators extend the Factory within their project context in such a way that the result is suitable for other projects, and for integration in new Factory releases.
     
  • We provide one-click functionality to any Factory user to share guidance on any Factory action, or to post remarks or requests to clarify guidance. The community contributions are then used to create or improve the approved guidance. This way we prevent a central documentation bottleneck and we prevent work on guidance that is not needed.
     
  • The Factory IDE is versioned with each product that is built with it; multiple Factory versions can live side-by-side on the same machine. This way, factory upgrades and customizations can be done independently for each solution, even when the solutions are developed in shared DTAP environments. The impact of Factory upgrades is much less, because you only upgrade for projects if and when the benefits are significant in the current lifecycle stage of the solution.

Closing remarks

I hope this post gives you an impression of where we have come from and where we are heading with our Factory approach. For me it is a privilege and a joy to be realizing this vision.

I would like to take this opportunity to thank everyone who helped to make this possible. In particular I would like to thank Maarten Sikkema for his unwavering belief and support to let us realize this vision, and his personal, hands-on involvement. Also many thanks to Serge van den Oever for the great teamwork and his many innovative ideas. Thanks to the Factory Champions team who encouraged and helped their colleagues to use the Factory and who provide us with invaluable feedback and direction. Thanks to the innovators who volunteered to integrate their learnings in the Factory, and last but not least thanks to all Macawers who believe in and use the Factory.

Moving forward, realizing tomorrow’s IT, today, the road ahead beckons brightly. In this article I outlined the vision that resulted in the Factory as it exists today. More recent developments in Microsoft IT have inspired us to expand our vision. Specifically VS Team System 2010, Microsoft Azure, GoGrid, ElasticHosts, and textual and graphical modeling with M and Quadrant offer great potential for future Factory versions, to raise the abstraction level, target cloud solutions and offer factories in the cloud, and to target more team roles and application lifecycle management (ALM) aspects. But more on that at a later time…

 

Vincent Hoogendoorn

About the Author
Vincent Hoogendoorn is a hands-on .NET architect and a Xamarin cross-platform mobile veteran, Microsoft Most Valuable Professional in Developer Technologies, and works as lead mobile @ Workstreampeople. His areas of interest are Augmented Reality, Cross-Platform Mobile Development, Windows Azure, Solution Architecture and Software Factories.
  1. Pingback: VincentH on .NET » Blog Archive » Factory Overview, Part 1: A Bird’s Eye View of the Macaw Solutions Factory

Leave a Reply


*

captcha *