Tag: DevOpsDay

Introduction

I am delighted to attend DevOpsDays NYC 2020 held on March 3 – 4, 2020 at the New York Academy of Medicine . I wanted to share about my experience for the 2 days, so we go.

• 

Day #1

The location was sort of far in the upper town of Manhattan from where I live, Brooklyn. However, after getting off the subway, the walk to the venue through Central Park was nice.

• 

• 

I arrived around 10am and only attended the last bit of this part before coffee break.

Next was a series of Ignite Talks which comprised of multiple 5 minutes talks. The key takeaways from these are:

• We should modularize notebooks for productionalizing data science models. Make them maintainable using modules and versions. Decouple and specialize child modules
• Incorporate security tools into CI/CD pipeline
• Resilience engineering is a community, and also means we should adapt to changes and learn from other industries such as medical, aviation, etc.

• 

After the Ignite Talks was lunch. It was fine apart of the part that it is lacking of vegan options. I ended up having pasta with cheese. 😐

• 

After lunch was Open Spaces where attendees get to suggest the topics that they want to talk about. There will be a subject matter expertise to facilitate each talk.

I selected these 3 topics and with the following takeways:

DevSecOps

• Bake security into process and tools
• Automate as much as you can
• Secure driven development – use tools to check flaws in security
• Have someone as security champion in the team
• Plug in security checks early, before pull request

Kubernetes

• Do not use multi-cloud, use namespace instead
• kubetl weakness – async and does not know when it finishes
• Use security tools to scan images

Observability/Logs

• Use distributed tracing
• Cloudwatch
• Key in tarce id
• Logs in json format
• Incorporate tracing before going live
• Use auto-instrumentation
• Use open source tools
• Use industry standard
• Incorporate into pull requests
• Tracing platforms (APM) are like DataDog, NewRelic, Elastic APM

Key takeaways from afternoon sessions on CI/CD Agility and Controlling Pipeline Sprawl by Angel Rivera.

• Avoid clear text in CI/CD
• Use tool like Hashicorp Vault to protect passwords
• Use random password generator to change passwords often
• Auto rotate the passwords
• Pipeline in YAML format
• 1 pipeline in 1 repo is not a good practice
• Do not hardcode in pipeline, use scripts
• Create vendor libraries for reusability
• Minimize vendor lock in

Day #2

• 

Key takeaways from Ignite Talks

• DevOps principle – has to have production mindset
• Is K8 really necessary? Automate everything, test twice, change architecture instead
• All tech is debt, people are gold – stop building new technology
• When software incident happens, mitigate or rollback 1st, learn from it, and practice (drills)

• 

Next were Open Spaces. I went to a salary negotiation, learning from software incidents and talk pay sessions.

Below are key takeaways from salary negotiation:

• Do not give a number in the initial interview process
• Focus on how you can give value to the company
• Have multiple offers
• Negotiate at the end of the interview process
• Its hard to negotiate in the same company

Learning from software incidents:

• Incident are operational surprises
• When there is a problem, implement more metrics and have processes in place to prevent the problem
• Test more
• Think of different ways a problem could have happened
• Learn from things that did not fail, how we did it right

Open space #3 was interesting as it had attendees to enter their base salaries based on dev, ops, or others (qa) regardless of experience levels. This session had the most attendees for obvious reason.

• 

The ranges vary widely from 5 to high 6 digits.

Key takeaways from afternoon sessions:

• Product management is customer focus, provides strategy + vision, allignment+leadership

• Product = Customer * Business * Technology
• Product managers gathers requirements, syhnthesize feedback, prioritize against business goals and broadcast value
• Name your services and be specific, says what it does
• Version your API, have clear documentations and examples
• Update runbook regularly
• Alerts for SLO level
• After alert is triggered, tune it, see patterns and prune
• All alerts should be actionable
• Need to understand business impact of the alerts
• DevOps should be low context, carefully constructing defaults, have ubiquitous documentation, document as much as you can

• 

It was a very productive conference as it is relevant to what I do. Looking forward for another DevOpsDay!