Process Implications
1. When two simultaneous preflights are trying to test changes to the same file and attempt to auto-commit, what happens?
This behavior depends on the policy set in the customizable preflight driver scripts. The first preflight to reach the auto-commit phase is able to successfully commit. If the SCM supports auto-resolution forb merge-conflicts and the appropriate policy was set, the second preflight may attempt to sync the file to the latest changes and auto-resolve any conflicts before checking in. This raises the risk for a broken build because both changes in combination may cause unwanted behavior, which is why you could turne off this policy. You may decide this is an allowable risk because the preflight mechanism dramatically decreases the number of bad check-ins.
2. If a preflight build fails, how do you obtain access to the files for testing/debugging? Does the machine [ec:running the build] stay reserved for further study?
All build information is stored in the assigned workspace(s) for the Flow job. Whether the machine stays reserved depends on the build process. It is easy to add a simple customization [ec:using properties] so a developer can reserve the machine until testing and debugging is finished.
3. Can developers run incremental builds while running a preflight build?
Yes, but it depends on how the incremental build system was set up and may require some customizations.
4. Can I use Flow to schedule a preflight build?
No. Preflight builds must be started from the developer's machine because that is where you also have access to modified sources.
5. Our developers run builds on their own machines already, so why is preflight better than that?
When developers make changes, they generally build and test their code locally and commit their changes if the code ran successfully. Unfortunately, this process is not always sufficient. Code changes may break the production build because of environment differences, platform-specific issues, or incomplete commits. Preflight builds address these issues because they run a simulated build as if the developer had already checked in the code.
Also, preflight guarantees traceability to see if a desired process is being followed correctly by development team members. To date, experience shows preflight benefits eventually encourage all developers to use the preflight build process regularly when it is available to them.
6. We were considering Flow to implement a continuous-integration approach, so why is this better?
With preflight, build issues are observed sooner in the process, segregated from other developers. The build failure impact in such builds is limited to the individual running the build only, and not other developers.
You may choose to implement a Continuous-Integration approach with either a frequent set of builds (for example, hourly), or by directly monitoring your SCM system to launch builds after check-ins occur. However, in both cases, it is possible for multiple check-ins to occur. If a submission results in a build failure, this approach can cause repeat build failures without identifying the root cause. With preflight, most build failures are isolated to the developer who created the problem, permitting the integration stream from other developers to continue unhindered by these problem cases.
7. If auto-commit is not used, how does a developer know if the build finished?
Use email notifications at the end of the job to notify the developer of the result, or leave the web-page open [ec:in the IDE] to observe the job as it progresses.
8. How should development teams make changes spanning a series of files, with multiple users changing those files, be guided on how to use the preflight build process?
Because preflight builds are designed to validate check-ins in isolation [ec:without the risk of build breakages], the number of files and users does not affect the process.
If, however, several changes are made to the same set of files in a short period of time, developers must constantly sync their source tree and resolve conflicts. While this process may be time-consuming, it protects against build breakages.
An alternate approach would be to do check-ins to a separate (private) branch, then run preflights when the code is ready to be merged back into production. This way, individual developers can build and test locally before checking in new or modified code, but the production build will not break even if they check in bad code. The preflights are then limited to the phase where changes made to the separate branch are merged back to production.
9. Can a single user submit 2 preflight builds concurrently?
Yes
10. How does Flow know which files need to be copied over to the production build machine?
The preflight system queries the SCM system used by the user and collects a list of files that are part of their pending check-in. These files are uploaded to the Flow server so they can be overlayed on top of the source code snapshot.
11. Can the system handle more than one preflight build at a time?
Yes - assuming you have sufficient resources.
12. Can I do a check-in without a preflight build? If so, how do I enforce users to actually run preflight builds?
Yes, check-ins can be done without a preflight build because Flow does not control the SCM system. Our experience shows that developers prefer running preflight builds to ensure they are submitting stable code. Some SCM systems may allow enforcing check-ins by forcing users to enter a Flow job ID. Cross-reference reports may also be generated to see which SCM changes were submitted without appropriate job IDs.
13. What happens if two users submit changes simultaneously, pass their preflights, but their changes are incompatible with one another?
The exact behavior varies by SCM. The first user will successfully auto-commit, and the second user will then run into a merge conflict. At this point, we may try to auto-resolve the changes and check-in if there are no conflicts, depending on the SCM capabilities.
14. What happens if another user performs SCM updates between my preflight build and check-in?
There is no difference in behavior between this and the question above. The second user who attempts to auto-commit will have to merge the changes and resolve any conflicts before checking in (or running another preflight to make sure the simultaneous changes did not break the code).
15. Can the system update a bug/defect status automatically when it does the autocommit?
Yes, with customization - this function might already exist within your SCM utility. Preflight auto-commit requires a change description, so if you submit your defect number as part of your description, you could configure your SCM-checkin procedure to use this information to impact your bug tracking tool.
16. Can a developer keep working after his/her preflight build is scheduled?
When auto-commit is turned off, you can change the files as soon as the preflight build is started because a copy of the files were already submitted to the remote build machine by this time.
When auto-commit is turned on, any file modifications that were submitted for a preflight after they were uploaded will cause the commit to fail. This failure is because the changes used to run the preflight and the changes ultimately committed must be the same. The suggested way to continue to working in this case would be to work in a separate workspace altogether, or if the SCM supports the changelist concept (for example, Perforce), you could use a separate changelist for new changes. However, no changes can be made to the actual files that were uploaded for the preflight build.
IT Implications
1. Will preflight builds choke my production systems with all these "user builds"?
It is important to understand that a preflight process helps reduce the number of broken builds you encounter---so the overall cost of dealing with broken builds, uncovering the change responsible for the broken build, including the lost build generation time while collecting broken build information, needs to be taken into account. The preflight benefit, however, requires available CPU-cycles to run these additional builds. You may want to reprovision your production system according to your estimated need. If your build takes a long time today, you may want to consider implementing an Accelerator solution to help shorten the amount of time each build is "in the system", using build resources, which will help reduce potential conflicts as well as encourage developers to adopt the preflight process.
Virtualization can drastically reduce the provisioning cost for a production system that supports an entire development team using preflight builds. ElectricFlow integrates with virtualization technology from VMware and Microsoft for increased efficiency.
2. What if a developer wants to use his own desktop for doing a preflight build?
An agent needs to be installed on the developer machine, and your procedure needs to supply an explicit resource name to be used.
3. Do I need a license for every developer running preflights?
No. Because licensing in Flow is based on hosts, users do not count towards the license restrictions. If, however developers are using their own machines as agents to run preflights, each such agent will count towards license restrictions.
4. Will we need to purchase additional build machines?
This depends on your existing load, and the additional load created by preflight builds. You may be able to leverage existing spare cycles. The larger your team, the more economies of scale can be leveraged to reduce costs.
5. What SCM systems are supported?
Perforce, Clearcase, Subversion, and Accurev are currently supported. However, the preflight mechanism is designed to easily add new SCM systems and customize existing ones. More information is available in the Preflight section of the online help system, accessed from the Flow web UI.
6. What if my SCM is not on your list of those supported out-of-the-box?
Other SCMs are supported by customizations. With Flow's extensive integration, you can create your own SCM "hooks". Over time, additional SCM tools will be supported as the market demands.
7. Do preflight builds support ClearCase? Dynamic or snapshot or both?
Both.
8. How are changes uploaded to the server? Do I need write access to the shared workspace or the Flow file system?
The developer's machine only needs to be able to communicate with the Flow server. Changes are uploaded via a streaming port using the STOMP protocol.
9. Do I need a client-side installation to run preflights?
Yes - either an IDE plugin or a tools installation if you are running from the command-line.
10. How do preflight builds affect ElectricAccelerator?
You always want to provision your cluster to match demand - and we provide reports to help with provisioning decisions. Adding user-builds to your existing build-cluster will increase cluster-usage, typically during daytime hours, so if cycles are free at that time before preflights are implemented, your Accelerator cluster may not be impacted. Preflight builds that run on a cluster are more efficient than distributing a large number of high-powered machines at the desktop, which are not always effectively utilized. From the opposite perspective, rolling out a process to encourage preflight builds is more effective and popular if the build time is short. ElectricAccelerator can help reduce your build times by leveraging a build cluster, so the combination of these two solutions are a good approach to make your process more agile.
IDE Questions
1. Can I control Flow from the IDE plugin? Can I create new projects, procedures, an so on?
Yes. Because IDEs support internal/external browsers, you can simply login to the Flow web UI from the IDE.
2. Which Visual Studio versions does your IDE plugin support?
2005 and 2008 - NOT 2002.
3. Which platforms does the Eclipse plugin support? Can I use the plugin on a system that is different from my Flow server?
Windows, RHEL 4 & 5. Yes, you can use the plugin on a system different from the server.
4. If I do not want to use preflights, is the IDE plugin still useful?
Yes. You can launch any procedure from Eclipse or Visual Studio. If you do not specify an SCM, the plugins allow you to start Flow procedures.
5. Does the IDE plugin require users to configure it themselves? How do I push upgrades to the IDE plugin?
Yes and no... A useful configuration can be managed centrally, and user customizations can be applied as needed at the project level. Visual Studio documentation explains how you can centrally configure and "push out" to the enterprise. Eclipse now has a Knowledge Base article that explains how to populate the configuration globally. However, some local level of configuration is required - password, changelist, and so on. These activities may be controlled in Eclipse by using variables.
Comments