Transcript
This transcript was autogenerated. To make changes, submit a PR.
I'd like to start out my talk by dropping a little bit of a truth
bomb on you, and that's that. If there's one thing I've
learned in my career in tech, it's that we get to choose
a lot. Now, we may not like
what we're choosing things horror
shows from my past that I've had to choose between, and we may not like
why we have to choose any of these things at all.
And we definitely don't like how often we get to choose.
Don't bother reading the details of this slide. It's an eye exam that basically is
saying, shit changes all the time. In fact,
one of the things that I laughed about the most at the start of the
pandemic was all the media outlets saying, oh, it's the new
normal. And I'm thinking, oh, you mean when everything we know
gets thrown out the window and we have to find a completely different
way to do every aspect of our jobs? Yeah,
in tech, we have a word for that. It's called, oh, my gosh, look,
it's Tuesday. The sheer volume of
opportunities to choose creates a number of challenges,
but maybe none as much as the paralyzing fear
that we might choose wrong, which is the
point of my talk today. Thank you for joining me. As I look back across
a lifetime of poor technical choices
or another title for this is when you stare into the face of
failure and success, smiles back.
I don't know. It just happens. Thank you for joining
me. My name is Leonidato. I'm a middle aged white dude giving a conference
talk. I selected this picture to give you confidence that I know what
the hell I'm talking about. Nothing else on that slide really matters,
except that you can find me, as at Liana Dotto, on almost all
social media. More importantly, I want
you to understand that this is what I call an oyster talk. Now, I got
that idea from the parenting educator, Barbara Colaroso.
And the point is that I have about half an hour to
talk to you here, and I cannot teach you anything in that amount of
time. In fact, in that amount of time, about the only thing I can do
is irritate you. Now, when an oyster becomes
irritated, it does one of two things. It either creates a pearl
or it dies. Now, it is my sincere
hope that I do not irritate you to death today. However, I did think
it was fair to let you know that those are some risks.
Instead, what I hope is I give you just the grain of an idea
that you will then think about and layer over with your own experiences
until what you have is something that is polished and unique
and valuable to you. Now,
after 35 years in tech, I've learned something, and I want to share it with
you all right up front. And that's that. There is no such
thing as a poor technical choice. I know because I've
made some doozies. And today I'm just going to share a few
of the residents living in my technical house of horrors.
Okay, I take that back. There are some common decisions which
are clearly bad. For example, not having backups, that's bad.
Or thinking that replicas or mirrors or raid are backups,
that's also a bad technical choice. Moving fast and breaking
things in test, sure. In prod,
not so good. You do have
a test environment, right? Like we all should have a
test environment. We should all have an environment that we test things in that is
not actually production. Okay, you get my point. Bad technical
choice. Trying to do everything all by yourself. Bad technical
choice. Not a personal choice. A bad technical choice. Using oracle
for literally anything at any time can be considered a
bad technical choice. Starting a land war in Asia.
Okay, fine, that was a princess bride quote, but I still had to include it
there. All right, so let's start. In 1984,
case Western Reserve University, IBM, and at and T wanted to build the
world's first free online community. Now, just for context,
case Western is a university not known for its technical program.
IBM is the stodgiest computer company in existence.
And at T is the zombified remains of a phone provider that
got more or less legislated out of existence. What could possibly go
wrong with this plan? And unbelievably, they succeeded.
They created a free public community computer system
which bore a strong resemblance to the BBS software I downloaded
onto my dad's compuware pc and ran when he
wasn't home, until he got the bill after the first month, the telephone
bill after the first month, and shut that all down. Now, I spent a
lot of time on freenet for a few reasons. First of all,
it was free. Second of all, you could reliably download uu
encoded ASCII porn on it. And third, there was a one
limit, time limit on that BBS system unless you became
a moderator in one of the SiGs or special interest
groups. So I became an admin on the dungeons and Dragons Sig
because, of course I did. And I also became an admin on
the word processing Sig and a few others along the way.
Now, you'd have to be a child of the understand exactly how
much time you could sync into a dial up service. But you would
argue it's a complete waste of time, right? So freenet
was basically a glorified message board. And what you did on it was
that you posted messages on different areas and you
sent comes to the maybe 20 other people in the world who had
an email address in 1986. And to do that, you needed
to select an editor to actually write that stuff. Now, you had a choice.
You could use a homegrown abomination called Fred or
Emacs or Vi entirely at random.
I chose Vi as my default editor. I was 16
and I didn't know any better. And I learned Vi like it was
the most natural thing in the world. So,
fast forward to one of my first jobs. I was working desktop support in
an engineering company, and the Unix admin came down. He was looking
for one of us to train into supporting one of the new
systems. And so he sat us down. He was one of those typical,
you know, grumpy big beard and everything. And he talked
to us and he said, okay, well, I'm going to have to pick one of
you boneheads to do this. And even before I can teach you the system,
I have to teach you this weird thing that nobody ever gets, and it
takes forever for you people to learn. It's called Vi.
You know that scene in Jurassic park?
Yeah. The point is that without those hours I spent on
freenet, I wouldn't have gotten the chance to move up.
By the way, the system that I learned was bind version
eight, the first production version of what would eventually
be called DNS. There was no part of this talk where I say that learning
DNS was or is a good choice, just to be clear.
So next story, I got my start in tech as
a classroom instructor teaching adults how to use things like Wordstar
Lotus one, two, three. Word perfect. And my boss thought it was important that
I have some certification since I was coming from outside
the tech sphere. And so I spent 80 hours taking my word
perfect certified resource test, not studying for the test.
It took me 80 hours to take the test.
Back in my day, we had to certify 10 miles uphill
both ways in the snow, and we liked it anyway.
Where's word perfect now? I mean, it's kind of
part of comes, but not really, but bought by Novell anyway, it was a
complete waste of time. Right? By the way, you can see the certification right there.
Yeah, complete waste of time, right?
Well, Wordperfect had gone the revolutionary route of
hiding formatting codes before that, if you wanted to format something, you actually put
the formatting codes on the screen with everything else, but you would
see instead a representation of bold or underline or whatever.
Now, sometimes you had to fix overlapping codes, and for
that there was a feature called reveal codes. It looked like that.
So you can see down below in the second bottom half of my screen here
where there's underline and where those bold starts and stops and
all that stuff. Which is why a few years later,
it took me about 15 minutes to wrap my head around how HTML
worked. And that helped me in a lot of ways that I'm going to talk
about in a little bit. But let me keep going.
So, like you, my browser has about
8000 open tabs. But before the Internet,
the way that you would keep up to date on stuff is voraciously reading every
tech magazine. I would tear out articles that I wanted to keep.
I'd tape the pages into scrapbooks
and create my own IT technology murder board and everything.
And maybe worst of all, I really liked what I was learning
and so I wanted to share it with everybody I knew. And so I would
try to engage my coworkers in conversations. Did you see how Borland
is taking on the limb standard? I was really a
lot of fun at parties, let me tell you. I was so desperate
to share the information that the Unix
admin, the one who I knew Vi, and so he took a liking to me.
Well, he knew I knew Vi and he knew I learned HTML pretty
quickly. So he set up a web server for me and so I could
type the snippets of the magazines into web pages and
share them. Now, literally nobody ever read those pages,
but I learned how to create web pages really
quickly. And when I was out of work a few years later, I had
everything I need to start building websites for people on the side and
do it faster than most people could. Now, I'll explain more about that in a
minute also. But knowing HTML literally helped me put food on
the table. But there's more.
Typing HTML week after week got tedious, so I asked the
Unix admin how to automate it. He taught me literally enough
perl to be dangerous. I wrote ridiculously complicated
scripts that would take the raw text input out of a text file
and then smash it together into web pages and figure out where the line breaks
were and add the paragraph marks and everything and figure out what the next page
name was supposed to be and then update the index HTML file.
I was very, very proud of my achievement.
But like I said, nobody ever read those pages. So what
value did any of that have? So my leap
into the world of monitoring started with a tool called Tivoli.
Now later on, I would include tools, everything from
openview to Nagios, to SolarWinds, to new
relic to Shinkan to Zabix and Grafana.
But I have to admit, Tivoli was my first tool
for its time. Tivoli was a comprehensive multi million dollar
solution. It had modules for system inventory and software distribution
and metrics collection and event correlation.
Tivoli was also basically a gang of perl scripts and
a trench coat and that lobbed monitoring data
into a Corba backend database. Nobody with an ounce of common
sense wanted anything to do with this freak show of a tool. But of course,
I heard the word perl and I was all in. And just like
when I was 16, learning Vi, I was too new to everything to understand
how truly awful it was. It all seemed perfectly normal.
But everything I learned set me up for the next 25 years
of my career. Now, Tivoli was good
at a lot of things. Windows was not
one of them. So we had to find this weird third
party tool called SMS, which you see on the screen.
But it's not actually the Microsoft SMS it was. Before that,
it was just SMS systems,
server monitoring system. And integrating the data
wasn't just hard tools at that time not
only didn't easily integrate
or combine themselves with other tools,
they actively resisted the
effort of integration. It took months to get this
all to fit together. And in all honesty, it was a bad tool and
we probably should have built it ourselves at the time. But we learned a
lot about integration and data types and transforms.
And believe it or not, I use everything that I learned back at that time,
even today as I'm working with newer and more robust
tools. And yes, that SMS was bought by Microsoft
and turned to Microsoft SMS, which became SCCM,
which then became SCOM, which then became ECM, which then became whatever
the heck Microsoft is calling it this week. And again, knowing the
underlying foundations of how tools are built definitely helps.
Tivoli had a GUI for doing things like grouping
systems together for software distribution. And naturally you could create groups,
and you could create groups of groups, and then you could create a group
of groups of. Actually you couldn't do that because
every time I did that, the entire database would crash and corrupt
and I would have to either rebuild it from scratch or restore it from a
backup. And I really wanted to know why. And so I did some digging.
Remember that Corba backend that I talked about. It was one
of the first object oriented databases in history,
but it could only handle three levels of containership, of object
containership. After that, it would just die. Now,
Corba wasn't my choice. It was Tivoli's choice when
they. But the tool. But over the three separate weekends
that I spent rebuilding the Tivoli database, I learned a lot about the impact
of downstream consequences and also the importance of
understanding the architecture that was being implemented and knowing
the ramifications of that. Because sometimes a software tool's pedigree
really does matter. Okay. The work that
I did on Tivoli got me noticed at the company I was at, which was
Nestle, and they offered me the chance of a lifetime. They were
going to move me and my family to the world headquarters
and work on a really big project, global monitoring.
250,000 systems in 5000 locations.
So they packed up my family and we landed in Switzerland
on September 1, 2000, and 110
days before 911. It was also six months,
almost to the day before the.com bubble burst,
at which point everyone in the project was sent home in the middle
of a recession. I was out of any
kind of steady work for 18 months.
I spent a lot of time feeling that that choice that I'd
made to accept the opportunity to move to a new country
and everything, I really spent time feeling that that choice had
ruined everything.
And over the next few months, I learned a little
bit about humility. I learned a lot about myself
and mental health and my value as a person
and not just the source of a paycheck. I learned who my friends
were and who I could rely on. And I learned
that times can get very, very tight
and still contain moments of joy and
how important it was to recognize those moments and
relish them and just be in
those moments, regardless of what's happening around you.
Okay. The next story actually wasn't me,
but I observed it very close up and I wanted to share
it with you. It's kind of poor technical choice. Voyeurism or rubber
necking, as the slide says. All right. I saw two
different sev one events. Events that brought
the entire system crashing down. Two events, one week apart.
In both cases, it was because of a configuration
change that happened in production. One person
was immediately walked out the door. The other person was given a stern talking
to. So what was the difference? The first person made
the change, saw that everything had crashed, immediately flagged
it, let people know, owned up to it. They couldn't
fix it. They didn't have the permissions or the skill, but they
offered to be there through the entire process. They offered to bring coffee,
to bring pizza, to watch, to learn,
to make sure that they understood the impact of
what they had done so that they would never do it again and that they
could actually help when they saw other problems that were similar happen
like it. The second person saw what
happened and tried to bury it. There's always
a log file. It's always in the logs. Now,
those lesson here should be pretty obvious. Do not try to hide your mistakes.
There isn't or shouldn't be the expectation of perfection,
but there is, or at least there should be an expectation
of decency. A few years ago I was tweaking an alert and
I accidentally caught 772
tickets twice in 15 minutes.
And the worst part was, I didn't even know that I had done it
until the entire help desk came up to my cubicle and invited
me down to their area to help them manually close
the 1544 tickets one
by one. Yes, there was a mass close function. They wouldn't let
me use it. Now, with those benefit of time
and experience, maybe some alcohol, I can reflect
on this formative event and wonder whose fault was it? Was it
the tool that allowed such a thing to happen? Or was it the operator?
There's probably enough blame to go around, but in a larger sense it
was an immature philosophy about what an alert actually is.
Now, all of that is the topic of a completely different talk. But for
today, the lesson is on the screen right there.
That screen was created by the vendor after the event
I just described, because I told them about it and said, why did you let
this happen? I wasn't the first person to experience it,
but when I became a Devrel advocate for that vendor
a couple of years later, it was something that I could articulate very
passionately and convincingly and get them to make a
meaningful change until they resolved it and gave a
screen that said, oh my gosh, you're about to do something bad. Maybe think
about it. So that's the point, is that the experience
I had fed into my ability to
make meaningful change. As I said at the beginning,
the sheer number of choices and possible negative
outcomes can be paralyzing. It is a Whitman sampler
of possible crap. In fact, you get this sort of choose
your own adventure kind of list of things. For example,
do you do kubernetes or do you choose not to do kubernetes? Do you
implement an integration to chat jeopardy. That is how you say it.
Or do you not do it? Do you use blockchain or not use blockchain
or this framework or that language or this platform?
Or do you put your system on AWS or Azure or
GCP or all three of them? Do you use oracle for
literally anything ever? No, you don't.
Do you use on prem versus colo versus cloud versus
a hybrid approach? Do you use monitoring with this solution or
that solution, or all the solutions combined,
or do you build it yourself? All of these choices
hold potential success or disaster,
and there's almost no way of knowing Seth
Godin, one of my personal heroes, addressed those inherent fear many
of us have. He called it technical fomo,
the fear of missing out, the fear of not getting into
or onto the next thing. And it causes us to hesitate
to get involved in something now. But what he said
is so very relevant. This thing, those thing we have
now, is worth working with because it offers so
many opportunities compared with merely waiting for the next thing.
It's always a good choice to work with
what you have now and not worry that you'll be
too engaged to pick up the next thing, which may be better,
but I know I said that my point was there were
no poor technical choices. But can
that be it? I mean, that seems really trite, really banal, really predictable.
Maybe the lesson is that the real treasure was the cables that
we collected along the way. Or maybe
it's something else. One last set of lessons.
I learned the word loquacious very early. Loquacious means
tending to talk a great deal. I learned it in fourth grade when
it appeared on my report card. Because my teacher
used it to describe me, I'd end up seeing that word a lot. Also overly
communicative and oversharing and diarrhea of
the mouth. I also wrote a lot
all through school.
Now, my notes were comprehensive, but not exactly
on topic. If you wanted to know whose chair made that weird farting
noise at 1115 in social studies class, I was your guy.
My notes may not have had the facts or details, but the 1812 war,
but in their own way they were comprehensive.
Also, from the time I was ten years old until I graduated
from college, 100% of my time was spent pursuing a
career in theater. In fact, my senior year I
had two academic classes and seven classes that
were choir, drama, band, and so on.
All that time spent, and I'm not doing any of that today.
So you could argue that all three of these things that you see were poor
uses of my time. But on the other
hand, I'm a developer relations advocate
or evangelist or whatever you want to call it.
And how do you do that job? Well, you do a lot of talking to
customers. You participate by typing in
online forums. You create videos like this one you
present at conference talks, you are with people on podcasts
and so on, along with everything you see here. Remember that
my entry into technology was to provide training to learn
how to relate to a group and present detailed information in an interesting
and even entertaining way, and sometimes to take a
technical pie in the face just to keep everyone enjoying themselves.
All that writing early on served me pretty well in the blogging circuit
and theater definitely helps me be comfortable in situations
like this where I'm standing up in front of an audience of people and sharing
ideas with you. Today there really aren't any
poor technical choices, and that's because it's
not about the tech. In fact, it was never about the tech.
It will always be about you.
I've left this slide blank on purpose to
leave room for what you bring to the story. You're probably already thinking about
your experiences and how they relate to the ones I've been sharing. You, or are
similar or different in many different ways.
We pick up technical puzzle pieces throughout our
journey in our career, and often it seems like the things that we pick
up have nowhere to go, but the truth is that they will
all fit somewhere eventually. In fact,
I felt so strongly that everyone's journey is valuable and worth
talking about that I made this talk open source. If you go
onto my GitHub repo, which you'll see there, you can download it and
submit it as a CFP to your own conferences, share your own amazing
details, and run with it. There is never going to be
a part of your skills, your experience,
your life that won't apply at comes point.
The trick the real lesson of this talk is that we have to commit
ourselves to showing up every day ready to
apply both who we are and what we know. And if
we do that, there really will be no such
thing as a poor technical choice.
As I mentioned at the start of this talk, I can't teach you
much, but I might irritate you. So if you're feeling irritated,
it's time to start mulling over that grain of an idea and creating
a pearl of your very own. Meanwhile,
online line in the chat of those
conference I am ready for your questions.
Thank you very much.