DORA 2024 Report

Last time I wrote about DORA Key Delivery Metrics and today I want to share key trends from DORA 2024 State of DevOps Report:

✏️ Artificial Intelligence (AI) Adoption. Report shows increasing AI adoption especially for the following tasks:
- Writing code
- Summarizing information
- Explaining unfamiliar code
- Optimizing code
- Documenting code
- Writing tests
- Debugging code
- Data analysis
While AI boosts individual productivity, it has been also linked to a 1.5% reduction in delivery throughput and a 7.2% decrease in delivery stability. It can be explained that teams can produce larger changelists , which in turn increases the complexity of deployments and the risk of failure.

✏️ Platform Engineering. Platform engineering has become a critical discipline for high-performing teams: teams that leverage internal developer platforms saw a 10% increase in team performance and an 8% boost in individual productivity. But at the same time there is 14% decrease in change stability. So platform engineering needs to be carefully implemented to avoid increasing overall pipelines complexity and stability.

✏️ Developer Experience. Report shows that focusing on the user increases productivity and job satisfaction, while reducing the risk of burnout. When this focus on the user is combined with an environment of internal documentation quality, this increase in product performance is amplified.

✏️ Organizational Stability.  Prioritizing stability in both technical and operational decisions can lead to higher team productivity and lower burnout.

✏️ Transformational Leadership. Transformational leadership is a model in which leaders inspire and motivate employees to achieve higher performance. A 25% increase in transformational leadership leads to 9% rise in productivity, reduced burnout and improved team and product performance. These leaders encourage their teams through the following dimensions:
- Vision. They have a clear vision of where their team and the organization are going.
- Inspirational Communication. They say positive things about the team; make employees proud to be a part of their organization.
- Intellectual Stimulation. They challenge team members to think about old problems in new ways and to rethink some of their basic assumptions about their work.
- Supportive Leadership. They consider others’ personal feelings before acting; behave in a manner which is thoughtful of others’ personal needs.
- Personal Recognition. They commend team members when they do a above an average job; acknowledge improvement in quality of team members' work.

The report highlights that transformation isn’t a one-time achievement but an ongoing process. Companies that are not continuously improving are actually falling behind. Companies that adopt a mindset of continuous improvement see the highest levels of success.

#engineering #news

Write Good Error Messages

"Cannot create entity X", "Connection to service Y failed", "Cannot read file Z".
These are typical error messages seen in many systems. And they are extremely bad.

So what's wrong with these messages? They are not actionable, they don't have any details what exactly went wrong (only for connection issue I can easily generate up to 10 reasons), they don't explain users or support engineers what to do next.

Good error message should:
- Be actionable
- Be detailed and clear
- Deliver the best user experience
- Enable users to help themselves
- Reduce support workload
- Speed up issue resolution

Google has a special chapter in their technical writing course about how to write good error messages. So let's check their recommendations:
✏️ Don't fail silently.  Failing to report errors is unacceptable. Assume that humans will make mistakes using your software. Try to minimize ways for people to misuse your software, but assume that you can't completely eliminate that.
✏️ Have a common style guide. Examples: Google API Error Handling, Go Error Handling
✏️ Do not swallow the root cause. Generic messages like "Server error" don’t help users understand or fix the issue.
✏️ Fail fast. Report errors as soon as they occur. Raising them later significantly increases debugging costs.
✏️ Identify the cause. Clearly explain what went wrong. Help users understand requirements and constraints. Be specific. Don't assume that users know the limitations of your system.
✏️ Explain how to fix the problem. Create actionable error messages. After explaining the cause of the problem, explain how to resolve it.

Example:
❌ Invalid input.
✅ Enter the pathname of a Windows executable file. An executable file ordinarily ends with the .exe suffix. For example: C:\Program Files\Custom\AppName.exe

Take time to train your team to write good error messages, it improves user experience, reduces support costs and speed up problem resolution.

#engineering #documentation

Data Vectorization

Let’s talk about one of the most fascinating parts of machine learning - data vectorization.

At first look, it seems like ML models work directly with the raw data we provide. But actually, it doesn't. ML algorithms need a numerical representation of data. Specifically, they require data in the form of floating-point values called feature vector.

However, many features are naturally strings or other non-numerical values. The task is to transform these non-numerical values into numerical ones. That's the main purpose of feature engineering discipline.

Let me illustrate that on vectorization of categorical data.

Categorical data consists of specific, predefined values, like car colors, animal species, days of the week, or city street names. It can be low-dimensional (few possible values) or high-dimensional (many possible values).

For low-dimensional data, we can encode it as a vocabulary. Let’s use car colors as an example with 5 categories for simplicity: white, blue, red, black, and others (for any color not in the list).

To create a vector the following steps should be done:
1. Index each value:
- 0: white
- 1: blue
- 2: red
- 3: black
- 4: others

2. Represent each category as a vector (array) of N elements, where N is the number of categories.
|Feature|White|Blue|Red|Black|Others|
|------------|---------|-------|------|--------|---------|
|White | 1 | 0 | 0 | 0 | 0 |
|Blue | 0 | 1 | 0 | 0 | 0 |
|Red | 0 | 0 | 1 | 0 | 0 |
|Black | 0 | 0 | 0 | 1 | 0 |
|Others | 0 | 0 | 0 | 0 | 1 |

3. Convert to floating point values: replace 1 with 1.0 and 0 with 0.0. For example, the vector for blue would be
(0.0, 1.0, 0.0, 0.0, 0.0)

Of course, that's very basic example to illustrate the concept. Real-world cases often involve more complex transformations and math models, depending on the data and problem.

More details and vectorization strategies:
- Working with numerical data
- Working with categorical data
- Datasets, generalization, and overfitting

#aibasics

Teams with high quality documentation are 2.4 times more likely to improve software delivery performance and meet reliability goals.

We all understand the importance of good documentation. But how often do you feel frustrated when trying to understand how a feature works? trying to find a solution for your problem? checking the source code instead of docs? Probably often.

Let's start with the difference between good and bad documentation. According to DORA, high-quality documentation should:
- Help users achieve their goals.
- Be accurate, up-to-date, and complete.
- Be easy to find, well-organized, and clear.

Tips to improve documentation:
✏️ Document Critical Use Cases. Clear use cases help users understand how to use your systems effectively.
✏️ Create Documentation Guidelines. When team members know when and how to make updates or remove inaccurate information, the team can maintain documentation quality over time. Check out Google’s Documentation Guideline.
✏️ Automate Guidelines Verification. Automate formatting and style checks with tools like Prettier.
✏️ Assign Ownership. Define clear responsibility and ownership for the documentation.
✏️ Define the Audience. Understand who will read your documentation, their background, and their goals.
✏️ Integrate into Development Process. Documentation should be part of your development process and stored close to the code. Documentation that is separate from development becomes dead right after publishing.
✏️ Automate testing for code samples or incomplete documentation.
✏️ Train your team. Teach your team how to write good documentation and explain why it’s important. I strongly recommend to check Google’s technical writing course for developers.
✏️ Recognize Documentation Work. Recognize and reward documentation efforts during performance reviews and promotions. Writing and maintaining documentation is a core part of software engineering work, and treating it as such improves its quality.

Writing good documentation isn’t easy. It requires clear processes, training, automation, and the right team culture. But investing in documentation improves your team’s development speed and overall delivery performance.

#documentation

Embeddings

In the Data Vectorization post, we learned that ML algorithms work with feature vectors. For example, we had a vector for car colors, where a blue car was represented as (0.0, 1.0, 0.0, 0.0, 0.0).

Imagine we create feature vectors for meal items in a dataset with 5,000 different elements. Each vector would have 5,000 elements, all set to 0.0 except for one element set to 1.0. This approach would require a high number of weights, a lot of memory and computational resources, making the model inefficient and hard to maintain.

To optimize this embedding techniques are used. Embedding is a projection of high-dimensional space of initial data vectors into a lower-dimensional space.

For example, in the meal dataset, we could introduce a feature like "sandwichness" and evaluate how likely an item is a sandwich. A sandwich might have a score 0.99, shawarma 0.9, and soup 0.0. But one feature isn’t enough, so we could add other dimensions, like "dessertness", "vegannes" or "liquidness," to better describe each item.

With features like sandwichness, dessertness, and liquidness, the vector for a hotdog might look like (0.95, 0.8, 0.0). Real-world models use many dimensions, but these vectors are much shorter and more efficient than the original 5,000-element vectors with only 0.0 and 1.0 values.

ML practitioners select dimensions based on the task they want to solve. This means that embeddings for the same items can be different depending on the context, task and provided data.

References:
- What are Embeddings in ML?
- ML Google Crash Course: Embeddings

#aibasics

UI representation of 3-dimensional meal classification from Google ML Crash Course

#aibasics

Happy New Year!

Another year has come to an end.

For me, it was a heavy and challenging year. But without a doubt, I’ve learned a lot, grown, and I am ready to move forward.

I wish you health and inner harmony. Take care of yourself, your family, your friends, and maintain a balance between work and personal life.

May the coming year be better than the last! Dream, plan, achieve, learn, experiment, love, move forward and enjoy every moment!

Happy New Year, Friends!🎄🎄🎄

Why Empathy Matters

Conflicts typically happen when our expectations don’t match with the actions or expectations of others. Here’s the key question: if the expectations are ours, why do we blame someone else for not meeting them? All people are different with different values, priorities, principles, background and life situations. And that’s fine.

Most conflicts might be avoided if we initially discuss shared rules that both sides agreed to follow.

But what should we do if we already missed that part and have a conflict situation?

Here are some tips to follow:
✏️ Focus on understanding, not blaming: Explain your position and try to understand the other person’s point of view. Operate with facts, not emotions and personalities. Look for win-win areas where both sides can come to an agreement and benefit from it.
✏️ Listen and show empathy: Carefully listen to what the other person is saying. Try to understand their motivation, accept their feelings and fears. Remember, that people always act in the best possible way for them. Show that you’re on their side, not against them. Often, people don’t hear each other’s arguments because there’s not enough trust or interest in each other’s opinions.
✏️ Work together to find a solution: When you have an understanding of both sides of the conflict, think about a solution that satisfies the requirements and interests of both parties. Start working together toward selected direction. Collaboration helps to establish trust and achieve better results.

Empathy is a powerful tool. It's more efficient for the business to reach an agreement rather than proving someone is smarter. Remember, conflicts can be very expensive.

#softskills #communications

Inspired: How to Create Tech Products Customers Love

"Developing a product mindset is essential for any engineer, and it becomes mandatory when aiming for an engineering leadership role." I read that in Hybrid Hacker newsletter about engineering leader roadmap some time ago.

So I decided to improve my product mindset and understand what product managers really do 😉 with one of the most popular book in that area - Inspired: How to Create Tech Products Customers Love by Marty Cagan.

Key thoughts from the book:
✏️ Product teams should have enough autonomy and wide set of responsibilities. Teams should have the ownership of the product they develop.
✏️ Product should have an inspiring vision. Vision is what business wants to achieve.
✏️ Product should have a strategy. A strategy describes how vision will be achieved.
✏️ Product teams should have correct business context to make right decisions.
✏️ Teams should use fast and cheap prototypes to test business ideas.
✏️ If the company uses OKRs, then they should be set for teams not for individuals.
✏️ Сontinuous product research and innovation are the foundation for the business success.

Principles for a good product vision:
✔️ Always start with Why. Define the main product goal.
✔️ Focus on the problem not the solution.
✔️ Be ambitious.
✔️ Be ready to break something old to build something new.
✔️ Be inspirational. Your vision should excite people, they should want to be a part of the product.
✔️ Be aligned with current industry trends.
✔️ Focus on the future.
✔️ Allow flexibility in details.
✔️ Require extra efforts to achieve. If it's easy to achieve the vision, it may be not ambition enough.
✔️ Evangelism. Regularly explain product vision to the teams and stakeholders.

Principles for a good product strategy:
✔️ Focus on one user profile or business niche at a time.
✔️ Align with the overall company's strategy.
✔️ Align with the marketing strategy.
✔️ Focus on customers not competitors
✔️ Share the strategy with the teams.

The principles and recommendations about product vision and strategy were the most valuable parts of the book for me. Other chapters felt a bit trivial - like using prototypes, making them cheap and fast, not doing real product from MVP, knowing your customers, etc. Anyway, the book is really good to improve understanding of product management.

#booknook #product #leadership

Word Embeddings

As we discussed embeddings can be different depending on the task, but one common task is predicting the context of a word. This is the foundation of how large language models (LLMs) work. LLMs don’t actually understand human language; instead, they understand the numerical relationships between words.

Key properties of a good word embedding:
✔️ Similar words should have similar vector scores.
✔️ Different words should have vector scores with values that are far away from each other.

When each word or data point has a single embedding vector, this is called a static embedding. Static embeddings, once trained, contain some semantic information, especially in how words relate to each other. It means that word similarity depends on the data the model was trained on.

To address the limitations of static embeddings, contextual embeddings were developed. Contextual embeddings allow a word to have multiple representations based on its surrounding context. For example, the word orange can mean a fruit or a color, so it would have a different embedding for each meaning depending on the context.

Tensorflow offers a great tool called Projector where you can play with `word2vec` static embedding model and see how trained data impacts words relationships.

Embeddings can be used for words, sentences and even documents. Interesting feature that is built on top of that - context search. Instead of searching by a particular word, you can search by the meaning of that word and find a related document even if there are no exact matches.

#aibasics

Large Language Models

In the previous post we learned how words can be transformed into feature vectors. However, language models don’t actually work with full words—they work with tokens. A token is the smallest unit of the model that can be a word, subword or even a single character (e.g., punctuation symbols).

For example, the word unpredictable can be broken into the following tokens:
- un (prefix)
- predict (root)
- able (suffix)

Language model is the model that estimates probability of a token or sequence of tokens appear within a longer sequence of tokens.

Example:
When I come home after work, I _________ in the kitchen.
Possible predictions:
- "cook dinner"
- "drink tea"
- "watch TV"

Each option has a different probability, and the model’s task is to pick the tokens with the highest probability to complete the sentence.

One more important element of the LLM is context. Context is helpful information before or after the target token. For example, context can help determine whether the word "orange" refers to a fruit or a color.

When a language model has a large number of parameters (billions of parameters), it’s called a large language model (LLM). LLMs are powerful enough to predict entire paragraphs or even entire essays.

Extending the example above it would be like:
User's question: What can I do in the kitchen after work?
LLM Response: _____

There is no magic - just math, probabilities, and predictions. So if you hear that LLM makes the scientific reasoning about some problem, it's not real reasoning, it's just advanced autocomplete for a given prompt.

#aibasics

Build Abstractions not Illusions

Creating abstractions is a key part of software development. We use abstractions at every level, whether it’s defining class hierarchies, setting component boundaries, or breaking down business processes. But how can we understand if our abstractions are good enough? That's the main topic of the Gregor Hohpe talk Build Abstractions not Illusions.

Key points from the talk:
✏️ Abstractions are needed to hide implementation details and reduce cognitive load for teams
✏️ Abstraction is the foundation of any model, models help us make better architecture decisions
✏️ Abstractions provide a higher-level vocabulary that hides underlying complexity. Abstractions should not be a composition of other elements (e.g., EngineGearWheelsAssembly instead of "car")
✏️ If an abstraction hides essential details, it becomes an illusion. Illusions are dangerous because they mislead users about the real system structure or behavior
✏️ Too much detail means it’s a composition; too little detail makes it an illusion. The right level of abstraction is a "gold" balance between these two.

The video suggests a really good model to think about abstractions in a scale composition - abstraction - illusion. A practical tip to check your abstractions is to ask: What details are essential? Can they affect the correctness of the model? If these details are missing, could that lead to misunderstandings about the system's properties?

#architecture

Small Language Models

While large language models (LLMs) are booming and discussed everywhere, there’s another important trend in the AI world—Small Language Models (SLMs).

A small language model (SLM) is a language model similar to a large language model but with a significantly reduced number of parameters. SLMs typically range from a few million to a few billion parameters, while LLMs have hundreds of billions or even trillions. For example, GPT-3 has 175 billion parameters, whereas Microsoft’s Phi-2, an SLM, has just 2 billion.

Main techniques to train SLMs:
✏️ Knowledge Distillation. A smaller model (the "student") learns from a bigger, already-trained model (the "teacher"). The student model is trained to not only match the teacher model’s predictions but also mimic its underlying process of reasoning. Typically the teacher model’s weights are frozen and cannot be changed during the distillation process.
✏️ Pruning. This is a process of getting rid of the extra bits that aren't really needed, making it smaller and faster without loosing too much accuracy.

Key advantages:
✔️ Resource Efficiency. SLMs are more compact and require fewer resources that makes them suitable for deployment on small devices.
✔️Cost-Effectiveness. They are much cheaper to train and deploy compared to LLMs.
✔️ Customization. SLMs can be fine-tuned on specific datasets, making them highly efficient for specialized tasks in particular industries.
✔️ Security. SLMs can be deployed locally or in private cloud environments, keeping sensitive data under organizational control.

There’s no one-size-fits-all solution when it comes to AI adoption. Every business will focus on efficiency and select the best and most cost-effective tool to get the job done properly. Architects should carefully select the right-sized model for each project based on its goals and constraints.

#aibasics

SLMs evolution timeline from SLMs: survey, measurements, and insights publication.

From the latest news Microsoft already released SLM Phi-4 model (Dec 2024). So SLM domain is under active development.

#aibasics #news

Radical Candor

One of the most useful and practical books that I read last year - Radical Candor: Be a Kick-Ass Boss Without Losing Your Humanity by Kim Scott.

Kim Scott is really experienced manager: she led teams in Google, taught general management at Apple University, coached CEOs at Dropbox, Qualtrics, Twitter and other tech companies. So she knows what she is talking about.

Let's start from a sketchnote that I prepared for the book to memorize key ideas. In next posts I'll explain them in more details.

#booknook #softskills #leadership #communications #sketchnote

Radical Candor. Part 1: Build Relationships.

There is nothing more important then listening to your people. It's not just additional responsibilities to the leader's job, it is the leader's job.

Trust relationships in the team are based on the following principles:
✏️ Care Personally: Show real interest to the people you work with, offer help where it's needed, support their growth. Remember, people have life outside the work, and this life can significantly impact their work.
✏️ Challenge Directly: Be demanding of yourself and others in a constructive way. Provide a feedback even it's negative. Stay focused on the issue, not the person.  Encourage high standards and accountability to show your commitment to the results.

Care Personally + Challenge Directly = Radical Candor 

Using these 2 dimensions the feedback can be:
✏️ Ruinous Empathy. The most frequent situation: avoiding tough conversations out of fear to hurt someone's feelings. It leads to cultures where nothing gets done, critical information is lost because people don’t speak up.
✏️ Manipulative Insincerity. Lacking both care and directness. It's usually an attempt to get the profit using emotional state of another person. Such interactions leave unpleasant aftertaste. Staff no longer cares about each other or the team's results. This is cultural bankruptcy.
✏️ Obnoxious Aggression. Criticism without care. The negative experience that person goes through usually ends up rejecting the message you were trying to explain. It leads to toxic work cultures.
✏️ Radical Candor. Challenging and direct feedback with empathy. Criticism is provided with the care and a wish to help. Don't hide problems - solve them together.

It's very important to follow radical candor principles to built health team culture.

#booknook #softskills #leadership #communications

My illustrations may be not ideal, but the concept is very important, so there is a more classic matrix representation from the book

Radical Candor. Part 2: Managed Growth.

I don't believe there is any such thing as "B-player" or a mediocre human being. Everyone can be excellent at something.

The author suggests to evaluate team members based on two dimensions: performance and growth.

If overall performance is positive then employee is one of two types:
✔️ RockStar - people with gradual growth, they value stability, knowledge sharing, attention to the details.
✔️ SuperStar - people with steep growth, they quickly acquire new skills or rapidly develop existing ones. Superstars constantly increase their set of responsibilities and influence within the team.

If performance is negative, then it's the leader's job to understand what's wrong (you can check how to do that in Why Don’t People Do What You Expect post). In some cases the best solution is allow the person to leave the team or company.

One really important thought that I hadn't considered before is that these labels (rockstar, superstar, low performer) are not permanent and can change over time. For example, when I started to work in IT company right after university, I spent a lot of efforts to my career growth. Then, I had a child, my priorities changed. I needed stability, flexible schedule and enough free time for the family. Later, when my kid grew older, I was ready to take new ambitious and challenging tasks again.

That's why it's so important to know your team and their current life priorities. A leader's job isn't about giving the meaning for the work but to understand what meaning each employee seeks in it. Only in that case you can help people to grow.

#booknook #softskills #leadership #communications

Radical Candor. Part 3: Get Stuff Done

The final part of Radical Candor book overview is about making decisions (other parts: Build Relationships, Managed Growth).

The author suggests a framework to make decisions and support health culture within the team - Get Stuff Done Wheel. The wheel contains the following steps:
✏️ Listen.  You have to listen to the ideas that people have and create a culture where they listen to each other.
✏️ Clarify. You have to create space in which ideas can be clarified, to make sure these ideas don’t get crushed before everyone fully understands their potential usefulness.
✏️ Debate. You have to debate ideas and test them more rigorously.
✏️ Decide. Make a decision. Sometimes it's better to split debate and making decisions on 2 separate sessions.
✏️ Persuade. You have to persuade those who weren’t involved in a decision (stakeholders) that it was a good one so that everyone can execute it effectively.
✏️ Execute. Implement the decision
✏️ Learn. You have to learn from the results, whether or not you did the right thing, and start the whole process over again.

That’s a lot of steps but they should be quick. Not skipping a step and not getting stuck on one are equally important.

Of course, the book contains many more useful techniques and tools to work with people. I would say it’s the top book I read last year from soft skills area. I recommend it to everyone because a good feedback is the foundation of trust relationships in a team—both from the leader and from all team members sides.

#booknook #softskills #leadership #communications

AWS Well-Architected Framework

How do you know that application in the cloud is well-architected? And if you think so how to prove that? There is no simple answer. Each major cloud provider, like AWS, Microsoft, and Google, has its own guidelines for that, called Well-Architected Framework.

Well-Architected Framework is a set of key concepts, design principles, and architectural best practices for designing and running workloads in the cloud. Recommendations are optimized for particular provider and their services usage.

AWS was the first who introduced this practice, so I’ll explain the framework based on AWS sample.

AWS Well-Architected Framework is based on 6 pillars:

1. Operational Excellence. AWS defines the goal of operational excellence as an ability to get new features and bug fixes into customers’ hands quickly and reliably. It contains guidelines for automation, continuous delivery, observability, managed service usage.
2. Security. It's more or less standard set of security recommendations: implement required security controls, apply security at all layers, be prepare for security events, etc. On of the most interesting principles there is to keep people away from data. The idea is that if people don't need to get direct access to the data (everything is automated), then the risk of data breach decreases.
3. Reliability. It contains practices to keep system up and running: self-healing, redundancy, recovery procedures, autoscaling to fit increased workload, careful resource management. Most principles there I've already covered in Reliability blog post.
4. Performance. This pillar ensures that computing resources are used efficiently to meet system requirements. It includes experimenting with the hardware, choosing right storage infrastructure for the workload, using already tuned managed services.
5. Cost Effectiveness. It's defined as delivering business value at the lowest cost. Cloud financial management should be established: consumption model adoption, measuring resource usage efficiency, scaling resources down when they are not needed not to pay for them.
6. Sustainability. This pillar was added quite recently and it considers the importance of long-term impact of the business on the environment (I wrote about the trend in Renewable Energy Trend). AWS highlighted that environmental sustainability is a shared responsibility between customers and AWS.

To sum up, the Well-Architected Framework is a set of best practices and design principles for running applications on a specific cloud. However, keep in mind that cloud providers also want to sell more of their services, so it’s important to stay a bit skeptical about managed service recommendations and carefully check the efficiency and costs for your particular payload .

#architecture #engineering