Python Scripting for DevOps and Automation
In this guide, we’ll dive into Python scripting — a subset of programming where you write short, focused scripts to automate tasks, interact with systems, and process data. This article explains the key difference between scripting and programming, then walks you through essential Python scripting concepts like variables, conditionals, functions, and loops with hands-on DevOps use cases.
Technology Stack
In this article, we will cover the following topics:
- What is Python Scripting?
- Core Data Structures
- Networking and API Interaction
- Object Oriented Programming (OOP)
- File Handling
- Exception Handling
- Python Scripting Advanced [Hands-on Project]
What is Python Scripting?
To understand Python scripting, it’s helpful to first differentiate between general-purpose programming and scripting.
What is Programming?
Programming is the process of writing complete, modular applications using structured and reusable code. Programs are typically longer, follow software engineering practices, and are intended for large-scale use. Python, Java, C++, etc., are used in such environments.
Use Cases:
- Building web applications using frameworks like Django or Flask
- Creating data pipelines and backend APIs
- Developing ML systems or automating business workflows
Examples:
- A Django-based blog, an e-commerce site, or a dashboard for internal tools
- Netflix using Python for recommendation systems and real-time analytics
What is Scripting?
Scripting refers to writing lightweight programs (scripts) for automating repetitive tasks. Scripts are typically smaller than full programs, focus on a single task, and are often written in interpreted languages like Python, Bash, or PowerShell.
Use Cases:
- Parsing logs
- Renaming files in bulk
- Monitoring CPU or disk usage
- Triggering alerts or backups
Examples:
- A script that checks disk usage and sends an email if space exceeds 80%
- A one-liner to batch rename `.txt` files to `.bak`
- A Python script to read SolarWinds logs and notify the team on high CPU usage
Why Python for Scripting?
Python is a preferred language for scripting due to its:
- Easy syntax and readability
- Cross-platform support
- Rich standard library
- Support for automation, REST APIs, cloud SDKs, file handling, etc.
Essential Python Scripting Concepts
Intermediate Python Scripting
1. Core Data Structures:
What are data structures in Python?
Data Structure means how our Data is organized.
In Python Scripting, we have 4 most important data structures:
- List: [23, 10.23, "Server", "jack", 3, 76, 3] - A list can store various types of data, including integers, floats, or strings, and allows duplicates.
- Set: {1, 23, 12, 3, 40} - A set is similar to a list but does not allow duplicates and is defined using curly braces.
- Dictionary: {"server1": "aws", "server2": "azure", "server3": "gcp"} - A dictionary stores data in key-value pairs, making it easy to retrieve data from large datasets.
- Tuple: (2, 43, 5.76, "server", "linux") - A tuple is similar to a list but is immutable, meaning elements cannot be added or removed.
Example of Data Structures:
# List in Python environments = ["dev", "prd", "test", "stg"] print(f"Here are the environments: {environments}") print(f"1st index or the second env: {environments[1]}") print(len(environments)) # will print the length of the list environments.append("basiton") # will append an element to the list print(f"Here are the environments after append: {environments}") # Dictionary in Python device_info = { "name": "Ubuntu-machine", "RAM": "8 GB", "CPU": 8, "IsItNew": False } print(device_info.get("name")) # to get the value from a key called name print(device_info.get("RAM")) # to get the value from a key called RAM print(device_info.get("CPU")) # to get the value from a key called CPU print(device_info.get("IsItNew")) # to get the value from a key called IsItNew device_info.update({"env": environments}) # adding another item to the dictionary print(device_info) # Sets in Python device_id = {1, 2, 3, 4, 5, 12, 14, 15, 23} new_device_id = {1, 2, 3, 7, 9, 11, 16, 19, 20} print(f"Unique IDs from device_id: {device_id}") print(f"Unique IDs from new_device_id: {new_device_id}") print(device_id.union(new_device_id)) # shows unique IDs from both sets print(device_id.intersection(new_device_id)) # shows common IDs from both sets # Tuple in Python days_of_week = ("sunday", "Monday", "Tuesday") print(f"The days of the week: {days_of_week}")
Output:
Here are the environments: ['dev', 'prd', 'test', 'stg'] 1st index or the second env: prd 4 # Length of the list Here are the environments after append: ['dev', 'prd', 'test', 'stg', 'basiton'] Ubuntu-machine 8 GB 8 False {'name': 'Ubuntu-machine', 'RAM': '8 GB', 'CPU': 8, 'IsItNew': False, 'env':
['dev', 'prd', 'test', 'stg', 'basiton']} Unique IDs from device_id: {1, 2, 3, 4, 5, 12, 14, 15, 23} Unique IDs from new_device_id: {1, 2, 3, 7, 9, 11, 16, 19, 20} {1, 2, 3, 4, 5, 7, 9, 11, 12, 14, 15, 16, 19, 20, 23} # Unique IDs {1, 2, 3} # Common IDs The days of the week: ('sunday', 'Monday', 'Tuesday')
2. Networking and API Interaction:
As a DevOps Engineer, if we want to use any API created or developed by a developer to connect to the internet, we use the requests module.
To install the requests module, run the following command in the terminal:
pip install requests
Example Code:
import requests demo_url = "https://jsonplaceholder.typicode.com/posts" # demo URL for JSON data response = requests.get(url=demo_url) print(f"Response from demo URL: {response}") print(dir(response)) # display available methods and attributes print(type(response.json())) # display the type of data received print(response.json()[1]) # fetch the 1st item from the JSON response
Output:
Response from demo URL:['__class__', '__delattr__', '__dict__', '__dir__', '__doc__',
'__format__', '__ge__', '__getitem__', '__init__', '__iter__',
'__len__', '__lt__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__',
'close', 'content', 'cookies', 'elapsed', 'encoding',
'headers', 'history', 'json', 'links', 'ok', 'raise_for_status',
'request', 'status_code', 'text', 'url'] # Type of data received {'userId': 1, 'id': 2, 'title': 'qui est esse', 'body': 'est rerum
tempore vitae\nsequi sint nihil reprehenderit
dolor beatae et ...'}
3. Object Oriented Programming (OOP):
In this section, we will explore how to code with classes, which is essential in DevOps.
What is a class?
A class is a collection of data members and member functions. It allows us to create functions and variables inside the class.
Example Code:
import os # Blueprint or a Template class Utilities: def show_disk(self): print(os.system("df -h")) def show_ram(self): print(os.system("free -h")) def show_sys_details(self): print(os.uname().sysname) machine_A = Utilities() # object 1 machine_B = Utilities() # object 2 machine_A.show_disk() machine_A.show_ram()
Output:
# Output will vary based on the system Filesystem Size Used Avail Use% Mounted on /dev/sda1 50G 20G 28G 43% / total used free shared buff/cache available Mem: 8G 2G 4G 0G 2G 5G Linux # System name
4. File Handling:
File handling in Python allows you to read from and write to files. This is essential for logging, data storage, and configuration management in DevOps.
Example Code:
# Writing to a file with open("example.txt", "w") as file: file.write("This is a sample file.\n") file.write("It contains some example text.") # Reading from a file with open("example.txt", "r") as file: content = file.read() print(content)
Output:
This is a sample file. It contains some example text.
5. Exception Handling:
Exception handling in Python allows you to manage errors gracefully without crashing the program. This is crucial in DevOps for maintaining system stability.
Example Code:
try: # Attempt to open a non-existent file with open("non_existent_file.txt", "r") as file: content = file.read() except FileNotFoundError: print("Error: The file does not exist.") except Exception as e: print(f"An unexpected error occurred: {e}")
Output:
Error: The file does not exist.
Python Scripting Advanced [Hands-on Project]:
What will we do in this Advanced Scripting?
- 1. We will take a backup of a server.
- 2. We will connect our Python with AWS account.
- 3. We will store the backup in AWS S3 (AWS service).
- 4. We will create a Flask app to monitor our server.
About the Author
Written by Omkar Goswami. Feel free to connect with me on LinkedIn for any queries or discussions.
For more reference Python scripts, check out my GitHub: GitHub Repository.
Conclusion
This article has provided a comprehensive overview of Python scripting for DevOps. By mastering these concepts, you can automate tasks, manage systems efficiently, and enhance your productivity as a DevOps engineer. Embrace the power of Python and take your automation skills to the next level!
No comments:
Post a Comment