TECH.insight

iOS continuous integration

Tuesday 26 August 2014

Photo by dariorug / CC BY

Continuous Integration is an important step in the software development cycle

We all want an “easy button” when it comes to continuous integration (CI) and deployment—a solution that requires minimal set-up and maintenance but that will complete all of the necessary tasks: Integrate and build, automate tests, manage defect logs and build environment, and deploy.

While achieving a one-click solution is feasible, it comes at a cost to set-up and maintenance. There is the challenge of the hardware set-up: A machine that needs applications, plugins, user accounts, security, constant updates, and more. Then there is the ever-changing landscape of integration tools, frameworks, apps, and scripts. At the end of the day, you’re left with something that becomes difficult to keep up to date and that doesn’t fit the needs of all your projects without additional scripts or add-ons.

Despite the challenge, CI is an important step in the software development cycle because of the services it provides. No matter the methodology you subscribe to, it can streamline your process by automating other steps as identified above.

The solutions

The first challenge is how to integrate. Should you use Jenkins, Travis, or OS X Server + Xcode + Bots? Why is one better than another?

Let’s start with the most painful part of iOS development: Certificates and provisioning. Apple Xcode Server stands out in this regard because you link the Server to your Apple development team and it manages the dependencies for you. No encryption or confusion about which files to use or where they live—simple.

Second, there is the issue of additional scripts you want to run, or services you want to connect. Every CI solution allows for these scripts, but implementing them can be frustrating. Jenkins fails to provide a viable option, as any script actions must be pasted into the Jenkins interface. While you can duplicate a project set-up, you must still go through the same set-up process over and again for each new project.

It also means disconnecting that set-up and any scripts from your project. Travis enables custom install and build settings from within your project directory that run during continuous integration, but this set-up has to be repeated project by project.

Another common factor is ownership and company policy. Jenkins and Xcode Server are both internally managed and owned. Travis uses a virtual server. While a positive feature for many reasons, this can be a headache when it comes to company policy, and leaves many developers concerned about any issues that might arise (security, process, archives, or otherwise).

Why work with Xcode Server?

While still imperfect, Xcode Server has a growing number of pros, in addition to hitting on the above points:

Of course, there are also cons to the Xcode Server solution. Enterprise Distribution requires additional set-up, and custom scripts are still needed to run additional steps at the end of the build (e.g. automation and deployment). But on balance, this solution makes maintenance of our end-to-end CI process less complicated.

Setting up your Xcode Server:

Apple's documentation on this subject is helpful, but there are still a few obstacles to overcome in setting up the Bots solution:

Install OS X Server and Xcode

After installing and starting up, you will want to do two things: Set up Users and Developer Teams. How you set up Users is up to you, but you may want to limit some of your team to only being able to integrate, while others have admin control over the Server. As for the Developer Teams, you will use your Apple ID to sign in, and this will manage all your certificates and provisioning.

Set up repositories

Apple claims your developers can set this up on their own via Xcode, but I strongly recommend using a single account for integration, and setting up your repositories on the Server manually using that account. Make sure to add all repositories used (e.g. submodules or any others you might be pulling into your project).

Once they are added and tied to that account, you will be able to perform Git actions such as incrementing of build numbers, and it will ensure your Server machine has all the required permissions. Currently, it is still recommended to use the https protocol over SSH. Apple claims an update for SSH will be coming with OS X Server 4 in the fall.

Developer machine set-up

Add your Apple account and Repositories within the “Preferences\Accounts” menu.

Bot creation

Apple truly has made this easy: Provisioning, Permissions, and Certificates are handled by OS X Server if you are using a developer account. Just follow the steps from the Apple documentation. Once you have successfully created the bot, you can run it. This will run the integration and perform the test, and analyze and archive actions as you choose upon bot creation. On completion it can notify you, and you will be able to download the necessary .xcarchive and .ipa files.

Beyond the bot

Creating a bot and build is not the end goal of CI. The end goal is a workflow of steps to automate your process. For me, this process consists of creating the build, automating tests as needed, running task and bug management to create release notes, generating a simulator build, uploading it to a centralized location with a link and password, and, finally, deploying the build to our internal team.

The service used for each of these “post-build actions” is different, and will vary from company to company. You’ll have to integrate these items through a custom script that you call from the “post-build” archive actions for your scheme.

To enhance this process, I wrote a little Ruby Library to run all of these steps as prescribed in a YAML file for each project. This Ruby Library is independent of each project, but simple to utilize with all projects, and can be updated from its own repository.

Step 1: Set up the project CI settings in a YAML format file

actions:
        build_simulator: false
        generate_releaseNotes: true
        upload_testFlight: true
        update_buildNumber: true
    defect-management-service:
        project: "TEST_PROJECT"
    provisioning:
        provisioning_profile: "PROVISIONING_PROFILE"
        code_sign_identity: "iPhone Distribution: Company"
    distribution-management-service:
        distribution_lists: "Distribution-List"
        notify: true

Step 2: Add the post-build archive action

Utilize a shell script to trigger the Ruby script with the project settings:

ruby "ruby-script.rb" "$PRODUCT_NAME" "$PROJECT_DIR" "$INFOPLIST_FILE"

Step 3: Basics of the Ruby script

The Ruby Library lives on the build server, and is triggered at the post-archive action from Xcode. It consists of a few steps and model objects that create the product and complete your CI cycle.

Build

Most of these steps will be replaced when OS X Server 4 is released, and we can use the resulting built product from the bot. You will also notice that much of this is put together quickly to string out bash script commands to carry out what we want. This does not need to be complicated.

def self.doPostBuildActions(project,product)
    # carry out project actions
    # the project.yml file includes a list of actions with a true/false and will carry out the specified actions
    project.actions.each do |key,value|
        case key

        when "build_simulator"
            if value == true
                BuildGear.buildSimulator(project,product)
            end

        when "generate_releaseNotes"
            if value == true
                project.setReleaseNotes(AdminGear.generate_releaseNotes(project,product))
            end

        when "upload_testFlight"
            if value == true
                # upload the ipa/dsym/details to test flight
                DistributionGear.distribute_testflight(project,product)
            end

        when "update_buildNumber"
            if value == true
                # increment the build number in the plist file
                project.incrementBuildNumber
                # commit the changed build number to git
                AdminGear.git_commit_buildNumber(project)
            end

        end
    end
end

def self.build(project,product)

    # Clear out any old copies of the Archive
    removeOldArchivesCommand = %Q{/bin/rm -rf /tmp/Archive.xcarchive*}
    Console.pipe(removeOldArchivesCommand)

    # Copy over the latest build the bot just created
    latestbuild = File.basename(Dir.glob("/Library/Server/Xcode/Data/BotRuns/*").max_by {|f| File.mtime(f)})
    copyLatestBuildCommand = %Q{/bin/cp -Rp "/Library/Server/Xcode/Data/BotRuns/#{latestbuild}/output/Archive.xcarchive" "/tmp/"}
    Console.pipe(copyLatestBuildCommand)

    # Remove old IPA
    removeOldIPACommand = %Q{/bin/rm "#{product.ipa_filepath}"}
    Console.pipe(removeOldIPACommand)

    runBuildCommand = %Q{xcrun -sdk iphoneos PackageApplication -v "#{product.app_filepath}" -o "#{product.ipa_filepath}" --sign "#{product.signing_identity}" --embed "#{product.provisioning_profile}"}
    Console.pipe(runBuildCommand)

    removeOldDSYMCommand = %Q{/bin/rm "#{product.dsym_filepath}.zip"}
    Console.pipe(removeOldDSYMCommand)

    zipDSYMCommand = %Q{/usr/bin/zip -r "#{product.dsym_filepath}.zip" "#{product.dsym_filepath}"}
    Console.pipe(zipDSYMCommand)

end

def self.buildSimulator(project,product)

    removeOldSimulatorCommand = %Q{/bin/rm "#{product.simulator_build_directory}"/"#{product.simulator_build_file}.zip"}
    Console.pipe(removeOldSimulatorCommand)

    runSimBuildCommand = %Q{cd #{project.project_dir};xcodebuild -sdk iphonesimulator -arch i386 CONFIGURATION_BUILD_DIR="#{product.simulator_build_directory}"}
    Console.pipe(runSimBuildCommand)

    package_simulator_build(project, product)

end

def self.package_simulator_build(project, product)

    FileUtils.cd("#{product.simulator_build_directory}") do

        Console.pipe %Q{mkdir "#{product.simulator_build_file}"}
        Console.pipe %Q{mv "#{project.product_name}.app" "#{product.simulator_build_file}"}
        Console.pipe %Q{mkdir "#{product.simulator_build_file}/Documents"}
        Console.pipe %Q{zip -r "#{product.simulator_build_file}.zip" "#{product.simulator_build_file}"}
        Console.pipe %Q{mv "#{simulator_name}.zip" "#{output_path}"}

        uploadSimulatorBuild(project, product)

    end

end

Admin

This “gear” will perform admin duties such as updating the Ruby Library, loading the .plist and YAML settings files, and kicking off the tasks for our project management service. In this case, I incorporate the jiralicious Ruby gem to integrate all of this with the issue and product-tracking software, Jira.

def self.generateFixVersion(project)

          jiraProjectID = project.projectid

          # HTTP BASIC AUTHENTICATION
          c = Curl::Easy.new("#{BASE_URL}/rest/api/latest/version")
          c.http_auth_types = :basic
          c.username = "#{USER_NAME}"
          c.password = "#{PASSWORD}"
          headers={}
          headers['Content-Type']='application/json'
          headers['X-Requested-With']='XMLHttpRequest'
          headers['Accept']='application/json'
          c.headers=headers

          c.follow_location = true
              c.on_failure do |response, err|
          end

          # generate date for release
          time = Time.new
          timeString = time.strftime("%Y-%m-%d")

          payload = "{\"name\":\"#{project.fix_version}\",\"project\":\"#{jiraProjectID}\",\"released\":true,\"releaseDate\":\"#{timeString}\"}"

          c.http_post(payload)
end

def self.generateReleaseNotes(project,product)

          jiraProjectID = project.projectid
    release_notes = "RELEASE NOTES: #{project.fix_version}"

          if (product.simulator_link)
            sim_note = "Simulator Build Available\n#{product.simulator_link}\npassword: #{product.simulator_pass}"
            release_notes = "#{release_notes}\n\n#{sim_note}\n"
          end

          readyForTestID = 101

          Jiralicious.configure do |config|
            config.username = "#{USER_NAME}"
            config.password = "#{PASSWORD}"
            config.uri = "#{BASE_URL}"
            config.api_version = "latest"
            config.auth_type = :basic
          end

          issues_remain = true
    while issues_remain
            result = Jiralicious.search("project = #{jiraProjectID} AND status = Resolved") # Any jql can be used here
            issues = result.issues
            total = result.num_results
            count = [50, total].max

            issues.each do |issue|
                #ADD FIX VERSION
                  currFields = issue.fields
                  currFields.append_a("fixVersions",[{"name"=>"#{project.fix_version}"}])
                  issue.update(currFields.updated)
            issue_note = %Q{#{issue.jira_key}: #{issue.summary}}
                  issue_note = issue_note.gsub "`" , "'"
                  issue_note = issue_note.gsub "\"", "'"
                  issue_note = issue_note.gsub "\\", "/"
                issue.save!()
              # TRANSITION ISSUE TO READY FOR TEST
                  Jiralicious::Issue::Transitions.go(issue.jira_key, readyForTestID)
              # GET RELEASE NOTE FOR THIS ISSUE
                  release_notes = "#{release_notes}\n#{issue_note}"
      end

        if total <= count
              issues_remain = false
        end
      end

      return release_notes
end

Distribution

This “gear” kicks off our service to deploy our build (using TestFlight in our example):

def self.deploy(project,product)

        command = %Q{/usr/bin/curl "http://testflightapp.com/api/builds.json" \
          -F file=@"#{product.ipa_filepath}" \
          -F dsym=@"#{product.dsym_filepath}.zip" \
          -F api_token="#{API_TOKEN}" \
          -F team_token="#{TEAM_TOKEN}" \
          -F distribution_lists="#{project.test_flight_settings['distribution_lists']}" \
          -F notes="#{project.release_notes}" \
          -F notify="#{project.test_flight_settings['notify']}"}
      Console.pipe(command)
end

Some common gotchas

For Enterprise Accounts (not required for Apple Developer Accounts):

For submodules: Even if you are not in a detached state, Xcode puts you in one. Make sure you access the submodule in Xcode’s Source Control and set it to the branch you want it to use, or you will have errors.

Something to be fixed when Server 4 is released: The “post-build actions” in your scheme are run before the build is available to you, so if you want things to happen, you still have to use xcodebuild.

You may also find these Ruby gems useful:

Summary

We all want an “easy button” when it comes to CI and deployment, and in my experience the OS X Bots solution I've detailed in this article is the closest I've found. I hope you find it useful.

About The Author

Kristen is a Senior Creative Developer at AKQA in Washington, DC, where she primarily focuses on iOS development. She started developing for touch screens 10 years ago in the museum space and maintains a passion for developing kiosks, installations and connected devices. She boasts a B.A. in Journalism - Visual Communication from the University of North Carolina.

@knov