Power of a Weekly Journal

One of the most challenging and important aspects of getting a new team off on the right foot is establishing good communications. These communications are essential to help everyone be connected to the initiative - whether that be an executive, an engineer on the project, a peer group that has a component to deliver, or a team that will be affected once the project is nearing completion such as sales or marketing.

Throughout my career I have seen lots of aggregated rollups of  manager weekly progress statements, summed up by an executive and distributed org wide. These are great, but often miss on tonality as they end up being giant lists of bullet points. We get lost in the details and miss the real storylines.

One of the best weekly summary authors I know is my former colleague Pritam Shah, who is now working on Google Cloud Platform. He really strove to bring his energy and passion into the summary for his team in the write up he gave. Initially it was just sent to 2-3 people outside of his team, but quickly it was forwarded to all corners of the company and his audience eagerly awaited his journal each week.

For my new gig I'm going to commit to capturing my thoughts and reflections in this model and helping to bring everyone else along for my journey. Each week I will cover a variety of different topics such as:

People & Collaboration

Product & Technology

Progress

Learnings

Challenges & what’s keeping me up at night

Aspirations

Since the journal will likely cover sensitive and proprietary information I won't be posting them here...but I'd love to see some examples. If you have any good ones, please send them my way!

Happy Journaling...

A New Gig

# A New Gig

Quick job status update: I have moved on from Lookout after 2.5 years and many amazing experiences that I may be writing about soon. My time at a security company was definitely quite eye-opening, and I have learned a great amount about the industry and the practices involved.

I am now involved in a new project for Delphix who has a really exciting and powerful Data Ops solution. My initiative will be bringing developing controls and management capabilities and delivering them via the cloud instead of on-prem. Lots of great stuff to come here and I'll share my thoughts as best I can!

Will we ever learn?

From Deming, Out of the Crisis:

Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.

And in today’s ultra-progressive workforce in software technology we get:

Early praise for Metrics Engineering

I’ve been working recently on a long form bit of material around metrics engineering and have a draft of chapters 1 & 2 completed. Check out the early praise!

“I’ll read it once I’m not so busy,” RHW 

“Aw honey, I’m so proud of your hard work,” TDR 

“Get back to work,” AG

I recently mentioned this would be a problem for Treehouse, another company experimenting with communal development. I will take this opportunity to say for the record, “I told you so.”

But as goes with the tree that falls in the woods, nobody will hear me now either :)

What I learned

I have come to the end of a 10 month experience that has been some of the most challenging time of my career and am extremely excited to be starting a new role at Lookout after a few weeks of much needed time off.

When I come to a transition such as this, I like to be reflective and think positively about the time spent. Here are a few thoughts to share:

-Being true to yourself and not compromising your integrity will lead to missing out on financial gain, and I'm ok with that

-Judge yourself on if you left the company in a better state that you found it, which I believe I did

-Some people you can't change, so if you really can't work with them, it's better that you find somewhere else

-If your job is to be a shock-absorber, you'll become emotionally depleted. It's a real thing.

-Your team is everything - whether it's your dev team or exec team - how they get along or disfunction will make all the difference in your outcomes

-In the age of technology, if you're too busy to stop and automate, you need to stop and automate. Same as the axiom if that you're too busy to meditate, you need to stop and meditate. Same goes for planning, be it strategic business or product planning.

-A company can operate with integrity and be successful, they are not mutually exclusive, as my next opportunity is proving.

-I really like technology platforms. My next role as VP Platform Engineering should be exactly what the doctor ordered for me.

Life of a platform

Here’s a way I have recently began to thing about the lifecycle of a platform. Really about the creation of a platform since the end of life portion for all coded systems is pretty much the same - dirt to dirt, electrons to electrons. In the beginning, software begins it’s life as a PROJECT. A PROJECT often comes into being as a solution to the needs of a specific client, or as an initial germ of an innovation that is seeking product / market fit. As it grows, a PROJECT can become an PRODUCT. A PRODUCT comes about when more than one customer or business wants to use the project and somehow folks have to figure out how to operate the germ of innovation that has been created. Finally it reaches the PLATFORM stage, when the desire to build more apps based on this thing that has been created succumbs to the notion of ‘let’s make it easier to build them by creating a PLATFORM.' So in this arc of PROJECT -> PRODUCT -> PLATFORM, it can be argued that platforms are the fully formed mature adults of the software ecosystem, soon to become the burden of younger generations who don’t want to pay for the legacy support to a greying populace*….ugh, make me stop. *I like social safety nets, this is just a joke.

Throw away your debuggers

Oh controversy, let me invoke you. On my team, developers have a habit of saying things like:      “How can I make sure my code works if I don’t have a debugger?”  Which leads to really complicated development environments where we have multiple vagrant instances that a developer can connect multiple debuggers to in order to test multiple services. This is a cop out. This is a crutch. Maybe not in all instances, but I believe it has lead our team to build code that isn’t transparent. And by that I mean the code running in PRODUCTION has piss poor diagnostics: logging and metrics. During development, the coders don’t worry about writing good log messages because they rely on the debugger to show them the internal state of the system. So writing good logs becomes another task, one easily de-prioritized, that QA doesn’t validate, that doesn’t really become apparent until something goes wrong in production. This is baaaad. So I say, throw away your debuggers. Use tools for development that you would use for production. Dogfood. Yum. PS. I know something debugging can be helpful. If the code isn’t yours, if you have poor unit testing, if you have really tightly couple code with complex interactions and states. I.e. the same excuses everyone has. But guess what...

What IS a steel thread?

I have recently had a running conversation with some colleagues, well, more of a disagreement, ok fine, a screaming argument, about what to call our first pass thingy that we built for a new project. The idea was to start building stuff that would help us understand the problem space for the vision of the new project. We were to explore both the technical and business implication of how we could build the system.   Two approaches emerged. One was called 'bottom up,' the other 'steel thread.' -The BottomUp-ers advocated letting the engineering team discuss, prototype and explore the anticipated biggest technical challenges first to ensure that our platform would be sufficient to meet the needs of the vision. -The SteelThreaders wanted to build an end to end, complete path through the system that could deliver value to our customers. Without going into a comparison of the two, it became clear that many people had no idea what was the purpose of a Steelthread. So perhaps more people are interested to learn that: A steelthread is Not a Prototype! As I mentioned above, a steelthread is an end to end system touching on all the logical and physical architectural components anticipated for the MVP. The idea of it is to prove to yourself how much effort it will take to deliver to your customer your anticipated value, as quickly as possible. You need to find out what it costs to deliver that value, what obstacles are in the way of delivering that and more value, and if in fact you are delivering the value you think you are. If you don't know what your MVP will likely be, then you probably can't make a steelthread. For example, if the vision for my product is to build a cloud hosted CMS for powering content on giant outdoor LCD billboard, then you had better hook up a giant outdoor LCD billboard to a simple app that can have editable content asap! Make is so that someone could pay you to post their content on the sign. Isn't this a prototype? NO! The purpose of a prototype is to learn so that you can build a better product. The purpose of a steelthread is to actually deliver value to your customers. Now, you don't have to launch a steelthread, this would be somewhat inadvisable because: A steelthread is NOT an MVP! (minimum viable product) Our toughest argument during this time was regarding whether we would throw away our code once the steel thread was completed. I let this misunderstanding hang in the air for a minute before clarifying: A steelthread is Not Throw Away Code! (one day I'll share my thoughts on why there is no such thing as throw away code, but not today.) Do not go into a steelthread process expecting to throw your code away when you are done. This is counter productive. Refactor it of course.  Decouple your components, of course. But purposely writing code to throw it away is not the point.

War stance

Appropriate in light of today’s news from CodeSpaces. If your business takes a warlike stance with regard to security, then there is no question about how much to invest in security, redundancy and service reliability. It’s your top priority. But if you do not put security #1, then deciding on the right amount of security is like deciding how much insurance you should carry. Once you need it you wish you had gone in for more.

Development Platforms, Hosted Runtimes and API services

I have seen many tech companies recently that don't quite know what they are  in the business of, and as such, often get their basic requirements wrong for who are their target customers versus who are their target users.  

One such area is when a company builds an application, sells it, then makes customer specific modifications. While this practice makes money, engineers often get frustrated because they want the business side to sell the application they have built, without having to support special customer needs. These needs often lead to a maintenance nightmare.  

So here's a quick recap of a few techie ways to think about your business:  

1) Application - most common in the consumer space. An application is about giving the same experience to all users of your system. If you want to be nuanced, then users of the same role all will have the same experience.  

2) Customizable application - entering the b2b space, many customers are more than just a single individual user, and these customers often have specific needs. One way to solve this is with a customizable application. Customers can either purchase access to different modules, or else manipulate the experience for all their users by changing application configurations.  

3) Development Platform - now entering the techie space, a development platform permits customers to write custom code that will alter the application experience for their users.   

4) Hosted Runtime - sometimes a development platform will also host the execution environment for the customer's custom code. But this is not a requirement for a development platform.  

5) API Services - last, many applications now integrate with API services that are hosted by another company, location, or environment. These APIs are not a development platform, even though you do development with them.  

Obviously there is much more involved in how to think about your business. But that's all for now.

Back in the saddle

After ExactTarget was acquired by Salesforce last year, I slowed down my public thoughts on this space. Now with my new team at Metrodigi, I hope to bring the conversation back to life.

Did I say conversation? That would imply something 2-way. This here is a monologue space according to Disqus...

Removing all the managers = removing all the accountability

In a post made a few months ago that I just came across, the start up Treehouse decided to remove all their managers in order to improve company productivity and culture. While their heart is in the right place, I can only wish them the best of luck in this adventure that will almost certainly fail.

Managers play a critical role in any organization. The post mentions a few important responsibilities (communications, dispute resolution, motivation, etc) and while they are admirable, completely miss the point of having a manager.

I believe the the most critical point of having a manager is to have clear lines of accountability. If everybody is responsible, then nobody is responsible.

At a small scale, you can find teams where the accountability is shared and everyone steps up equally to accomplish the jobs set out. But it takes time to craft, mold, and develop teams so that they behave like this. It takes leadership. This doesn't happen by accident.

By taking this step, the CEO has put all the responsibility on his own shoulders and can't delegate effectively because….wait for it….nobody has been assigned duties for a given area. Well let's carve out an exception for finance because financial operations needs to have some accountability. And whomever is assigned to that responsibility needs help, do you create a reporting hierarchy? What about HR? What about Marketing? Can anyone put out a press release just because they think it's the right thing to do? Or does everything need to be approved by the CEO? or is development special?

So yes, the blog title is the 'Naive Optimist', and yes, his views are quite naive…

Rarely do I read execute 'culture' write-ups that don't make my innards churn, but this is one of those occasions. I am really impressed that an executive can embrace these approaches, since it is what us on the engineering side really believe in:

Anyone wondering how Amazon Web Services is able to roll out so many new features to its cloud platform each year might just want to read the new biography on Amazon CEO Jeff Bezos, whose management style touches everything within the company.

This is by far the best spam ever! I have no idea what it even means for me to do...

Quick point on security in the cloud

There is a regular string of security breaches that make the public awareness for cloud products. Recently, Adobe had a big one, and this weekend Buffer had a breach that was due to a vulnerability in one of their cloud service providers.

Enterprise companies with software products need to really think through the security tradeoff. They need to classify security as a value protection task versus product development which is a value creation task.

The tradeoff comes for enterprises launching new products, where this is a very tricky balance since they have established security practices for the core products, but the new and experimental ones can be overwhelmed by the requirements and kill the innovation cycle.

Interesting look into how Yahoo has been working with Storm, including a dive into how and why they replace zeromq:

Yahoo recently submitted work to Storm that allows the messaging layer to be pluggable and provides an implementation based on Netty. This is an important step for many reasons. It enables Storm to run without dependencies on native library, and thus simplifies Storm deployment...