DEvOps
Connect with KAASM for news, updates, company culture, and technology trends.
Why DevOps with WinCC OA is cool!
In the IT world, you would NEVER, EVER, make edits to a production system without going through proper quality assurance change acceptance. However, in the industrial world, for some reason, it has been acceptable to test and commit changes on production systems without the need for isolated development testing prior to deployment. Shouldn’t we be asking, why?
Is it because…
- There is not enough time to set up this type of development environment;
- IT tools are not readily available for use;
- There is no awareness of DevOps tools or how to use them (know-how);
Understanding that IT and OT worlds are converging at an exponential pace, we need to help educate our industry about DevOps practices and the tools that are readily available in modern SCADA packages like WinCC Open Architecture. In our previous DevOps blog posts, we showed you how to integrate Jenkins and Git into your WinCC OA projects to facilitate automated unit testing.
In this blog post, we are going to bring it all together by setting up Continuous Deployment utilizing WinCC OA, Git, and Jenkins. This tutorial expects you to have the same set up as in the previous blogs and to have followed the previous tutorials to set up both your Git and Jenkins integration.
Requirements
- Git must be integrated into your WinCC OA environment (see: DevOps: 6 Steps to Integrate Git into WinCC OA)
- This tutorial uses a GitHub repo. You must have a GitHub account setup.
- Jenkins must be downloaded and installed on your development machine (see: DevOps: Integrating Jenkins into WinCC OA).
- Your Jenkins server and production WinCC OA server should be on the same computer.
Step 1: Create a class to hold your Unit Tests
WinCC OA does not natively interact with an outside system like Jenkins to manage development deployments and it is good practice to use something like Unit Tests to determine if your code should be deployed to a production system. I have modified the code from our previous blog post and created a new class to accomplish this. The major changes from this file to the automatically generated one from the High-Speed Programming module is that I am using the base level function of oaAssertEqual() to get the result of pass or fail instead of writing it to the project UI. I have two tests in this file and both are very simple and I know they will pass, in a real project these would need to be written in a way so that they would help prevent errors.
Step 2: Create a WinCC OA Web Server for Jenkins to hook into
We need some way for Jenkins to integrate into WinCC OA and a webhook is the easiest way to accomplish this. The following script will start a web server, make a single endpoint available, create an instance of the TstUnitTestDemo class from before, runs the two tests, and return a result.
Step 3: Add your server to your console
The easiest way to do this is to drag the file onto the Console and it will automatically add it. Make sure that you start the Control Manager.
Step 4: Update your Jenkins file
This Jenkins file has a little bit of logic in it mostly checking if the tests from WinCC OA pass or fail and if they do pass to run a git pull command.
Step 5: Set up a development environment
This is a little bit of a chore because not every file is stored in the Git repository, however, what worked for me was to copy the project to a different computer, deleting the .git folder, git cloning my repository, and then copying those files into my project. Git cloning will link your repository to your local project.
Step 6: Makes changes and push them to the repository
If everything is set up correctly when you make a push to your git repository Jenkins will recognize the change and execute your scripts. After Jenkins has run the scripts, if you look at your production system you will notice the changes to Scripts or Panels are now live on your production server.
Note(s) and Limitations
This set up is very limiting but exposes the power of these tools working together. I would also connect something like Slack to your Jenkins to notify your other developers that changes have been made. Also, this type of Continuous Delivery is only as good as the guidelines in your development team’s practices. If the team is lax on the Unit Tests this will not do you much good. Another thing to keep in mind is that the git version control set up inside of WinCC OA is built to track changes in panels, pictures, and scripts.
Read More
Over the past few weeks, we have been showing how to use DevOps practices and tools inside of WinCC OA. If you are new to this DevOps series of posts, feel free to check out these additional blogs:
What is DevOps?
DevOps: 6 Steps to Integrate Git into WinCC OA
Is TDD the Future of MES/SCADA Development?
DevOps: High-Speed Programming Unit Tests
Today we will take a big step further and integrate Jenkins into the development process. This tutorial expects you to have a few things setup and to understand a few others:
- Git must be integrated into your WinCC OA environment (see: Git integration)
- This tutorial uses GitHub. You must have a GitHub account setup.
- Jenkins must be downloaded and installed on your development machine (https://jenkins.io/download/).
- You must have written a unit test utilizing the High-Speed programming method inside of WinCC OA.
Step 1: Setup Jenkins to Connect to GitHub
The first thing you will need to do is generate a token for Jenkins to use. To do this, you will need to navigate to GitHub to generate a token at https://github.com/settings/tokens/. Create a new token and give it the rights (at minimum) to read the repository.
Once you have completed this setup, you will get a key that looks like this one:
Do not lose this key. You will not be able to get it again.
Step 2: Connect Jenkins to GitHub
Next, you will need to go into your Jenkins server and allow communication from Jenkins to GitHub. Inside the credentials options, you will need to add two new global credentials. One will be a Secret text and the other will be a Username and password.
For the Secret text credential, place the generated key from GitHub into the Secret input field and give it an ID of GitHubToken and hit OK.
Now complete this process again, but this time make it a Username with a password. In this one, you will enter your GitHub username and password.
Once this is done, restart your Jenkins server. Finally, we will establish a connection between GitHub and Jenkins. Click the Configure System from the Manage Jenkins menu option.
Scroll down the screen and find the GitHub server option. Select the Add GitHub Server dropdown, give it a name, and select the GitHubToken we created earlier.
Test the connection and then click Save on the bottom of the page.
Step 3: HTTP Request Plugin
We will be using REST calls to communicate with our WinCC OA instance. Adding in the HTTP Request Plugin makes it very easy to do this. Navigate back to the Manage Jenkins screen and click the Manage Plugins link.
Click on the Available tab and search for HTTP Request Plugin. I already have it installed so I am on the Installed tab.
Step 4: Open WinCC OA and Create a Server to Hook into Jenkins
For ease of use, a file will be included to make this step easier. What UnitTestServer.ctl does is creates a simple HTTP server inside of WinCC OA and open an endpoint for Jenkins to hit. The startTestsRecursive() function is a built-in WinCC OA function that is used to run the Unit Testing for the High-Speed Programming module. One note is that you will need to change the directory in the startTests() function to your own project name where your tests live.
You can download the file here: UnitTestServer.ctl
Next, take this script and add it to your WinCC OA console so it can be started.
If you are successful you should get the following message from the WinCC OA Log Viewer.
Step 5: Make a Jenkinsfile
In your Git repo make a new file called Jenkinsfile. It cannot have an extension. Place it at the same level as your .git folder. I make my Jenkins with two commands that do the exact same thing, but gives an example of how different things can be done at different build processes. Below is the code for a base Jenkinsfile.
Step 6: Create your Jenkins Pipeline
From your Jenkins server select New Item, then select MultiBranch Pipeline on the next screen.
Inside the Pipeline configuration, add the Branch Source GitHub. Select the credential dropdown to be your GitHub login credentials and enter in the repository that you added the Jenkinsfile to int he URL field.
Finally, change your Scan Multibranch Pipeline Triggers to run periodically and click Save at the bottom of the page.
Step 7: Make a Change and Test it Out
With both the Jenkins Pipeline and the UnitTestServer.ctl in WinCC OA running, make a change in your project and push the changes to your GitHub repo. In Jenkins you will see something like this:
In WinCC OA your Log Viewer will show this:
Final Thoughts
This is a simple example and one of the many things that can be done with WinCC Open Architecture, Git, and Jenkins. There are possible ways to streamline this even more with additional plugins and other ways to accomplish the exact same result. Think of this as a starting block that you can use to automate your process even more.
Read More
Enterprises throughout the world are rapidly starting to adopt DevOps strategies, but what does that mean? DevOps stands for Development Operations, and basically means the convergence of the Dev team, the folks creating the code, and the Operations team, those using the code. Historically, these two business units operate as silos, which has a negative impact on many factors within the organization. Some of these include slower dev times, slower time to fix security issues, and lack of collaboration between developers and users.
DevOps is really about creating a culture of collaboration and productivity by automating as much as possible, resulting in a faster time to market or more frequent releases. Enterprises that already have a DevOps mindset have a huge competitive advantage because of their ability to innovate faster, respond to business needs, and collaborate throughout the organization.
Optimal DevOps Flow Chart
There are many tools out there to help support an organization’s DevOps strategy. Below are the most commonly used DevOps management tools and a few programs within each category.
- CI/CD Management: Jenkins, Bamboo, TeamCity
- Source Code Management: GitHub, Bitbucket
- Configuration Management: Chef, Puppet, SaltStack
- Container Management: Docker, Kubernetes, Amazon ECS
- Development Environment Management: Vagrant, VirtualBox
- Code Performance Management: New Relic, Splunk
We’ll cover more of the above in future blog posts. What are some of your favorite DevOps tools?
Read More
As the role of DevOps becomes more and more prevalent in the world of automation and industrial SCADA/MES/IIoT software, one of the first steps that should be taken into consideration is setting up different levels of environments for your developers to utilize.
Why?
The easiest answer to this question is to prevent downtime from a poor deployment to production. Each level will add testing confirmation and check-offs that should be completed prior to the next step. The goal is to eliminate as many bugs or defects before your development is deployed into the actual production environment. If a bug is found in your test environment, the code can simply be sent back to the developers to fix it before letting it move on.
DevOps tools that can help
Automated Unit Testing
Make sure your industrial software supports automated unit testing in some way, this will help catch some bugs before it even reaches the next environment. An example of unit testing can be found in this blog: High-Speed Programming
Vagrant
Vagrant is a software that is used to create and reproduce identical environments so that you don’t run into the issue of ‘Works on My Machine’. There are two benefits of utilizing Vagrant:
- All the developers are running the exact same ‘stack’ of software
- Your DEV, TEST, PROD environments can be configured to be 100% identical
Git
Git is valuable in this set up to make sure that the code that is in one environment is in all the environments. When a developer makes a change and pushes it to a repository, all other developers can be notified of this change and can then pull down the code. This will help prevent two developers from having different code when trying to deploy it PROD.
BONUS: Version control!!
Jenkins or similar
Jenkins is an open-source automation server that can be used to help automate projects. Jenkins’ main purpose is to eliminate manual processes, consequently preventing errors from occurring, think of it as Automating your Automation. Below is a possible workflow using Jenkins and GitHub.
Downsides
Cost
The cost could be the major player in deciding not to use separate environments. Every environment will cost additional money to configure and implement, along with the development cost to run a proper process. On the flip side, depending on the critical nature or the scale of your project, this DevOps set up could prove to be invaluable.
Culture
Culture will be another major factor in setting up multiple environments. To keep these systems separate, gatekeepers need to be put in place to make sure a process is followed. Following a development process may pose its adoption challenges.
Even with these downsides, the benefit of preventing many downtimes should outweigh any possible complaints. To learn more about other DevOps topics check out our other blog posts:
Is TDD the future of SCADA/MES Development?
DevOps: 6 Steps to Integrate Git into WinCC OA
DevOps: High-Speed Programming Unit Tests
If you’re interested in KAASM setting up a test environment for you? Contact us for a Digital Transformation engagement.
Read More
WinCC Open Architecture (OA) has supported Unit Tests for quite a while now, however, when Siemens released version 3.16, one of the new features was High-Speed Programming. This new feature set included a tool for project documentation and the ability to auto-generate templates for faster creation of code. In this blog, we are going to look deeper at the new way to run and write Unit Tests inside of the graphical editor (GEDI).
Step 1: Create a Script
The first thing we need to do is create a script, or class, that we would like to unit test. I made a very simple script with two functions inside of it. If we were using a Test-Driven Development (TDD) methodology writing the functions would come later. Learn more about TDD
Step 2: Auto-generating Testing Script
Once you have the script created, all you will need to do is right-click that script and select the Open Unit Test option. This will autogenerate a WinCC OA Test class which is used to run the unit tests.
Here you can see the created script and libs.
Step 3: Define your Tests
When you open the autogenerated script you will see a group of code already written. The parts that will need to be added to are on line 28 and inside the startTestCase() function. Inside the makeDynString on line 28 you will need to add in names of the tests you would like to run. These tests should be small and only test a small unit of your code. I’ve added “add” and “getDataPoint”.
Step 4: Write the Tests
Now that the tests are defined it’s time to actually write them. Because we’ve added two entries above we need to add two cases to the switch statement. Inside of the “add” case, I did a simple assertEqual(6,result); . This means I am testing that 6 equals the result of 2 plus 3. This test will fail because this is not true. I also wrote a simple test that reads a datapoint and asserts that the value is what I expect it to be.
Here is the entire script.
Step 5: Run the Test
Now that the tests are written we can now run them by right-clicking the test script and selecting Start Tests from the options. This will cause the script and tests to run and display the results.
Step 6: View the Results
After we ran the test, from inside the GEDI we can see the results. We have two fail and one success judging by the thumbs up or thumbs down. The two that failed were the “add” and the “UnitTestDemo_”, which is a default autogenerated test that can be deleted.
For those that have been using Unit Tests inside of WinCC OA, none of this is to new because of the long support of Unit Tests in the platform but the new High-Speed Programming set of tools will make it easier for others to begin seeing the benefits of Unit Testing.
Read More
As SCADA, MES, and HMI projects become larger, more complicated, and interact with many different systems, the time has come to start thinking about adopting TDD and Unit Tests into your development lifecycle.
What is a Unit Test?
A Unit Test is simply a set of code that tests another set of code. Usually, Unit Tests are used in pair with additional DevOps tools to automate the running of these tests and to prevent deployments if one or more of the tests are unsuccessful.
What is TDD?
Test-Driven Development or TDD is a programming concept in which you write the Unit Tests for your project before you write the code that is needed to run them. An example of this would be, knowing that you need to add two numbers in your program. So you would write a test named something like test_add().
You would then run this test, and it would fail because you haven’t written code for it yet. Your next step would be to write the minimum amount of code needed to make this test succeed.
If you were to now run that same test above, you would get a successful test result. You can now go back into your add() function and refactor it by any means necessary but to still fall into good coding practices.
Downsides to Unit Tests and TDD
More time is usually needed to write good tests. This is one of the biggest arguments against Unit Tests and TDD but it should not be. Yes, more time is being spent upfront with the design of tests and functions but it also allows you to easily test if new code that is added breaks existing code by simply running a test. This makes it great for entry-level developers because it is easier for an upper-level developer to know at a moment’s notice if code is not working correctly.
Culture change. TDD is not a one developer task, the entire team must utilize this method for it to be completely successful. This is usually a problem because most developers just want to code and worry about “testing” later. This type of attitude will make this methodology pointless.
These examples were very simple and most likely not applicable to a real-world environment, but the concepts are very important, especially with SCADA/MES systems interacting with other systems in the IT realm. One thing you should ask yourself; if your IT level systems are expected to have automated testing attached to them, then why not your SCADA/MES system?
Read More