ELI5 – DES (kinda)

Uncategorized,

In my previous post, which was a review of the book Applied Cryptography and Cryptography Engineering, I wrote that DES, in spite of retiring officially (after numerous successful attacks), is still a great algorithm from an academic perspective to learn and peek into the minds of cryptographers. In this one, I’ll try to explain DES in my own words, testing my own understanding and giving you a (hopefully) nice read from a novice perspective. So let’s get started with DES, a cipher that was once the standard for government data encryption for the US and many others around the globe, now defunct and only exists, if at all, in form of 3DES.

Before we begin, let us understand where we are (with DES) in the cryptoverse and then talk about DES itself. In cryptography, encryption of data can happen in two ways. There’s symmetric cryptography, and there’s asymmetric cryptography. In symmetric key cryptography, we have block ciphers and stream ciphers. DES is a block cipher.

A Brief History

DES, for Data Encryption Standard, is a symmetric key encryption algorithm proposed by IBM with inputs from the NSA in 1974. Not many details about the development process were shared with the world. It is studied by numerous experts and counts as one of the most studied ciphers of all time. DES was designed to keep government secrets, secrets.

The 56 bit key size didn’t impress anyone back in the day, much less today. In spite of a small key size, there weren’t any attacks faster than brute force, both theoretically and practically, until into the late 80s when Adi Shamir and Eli Biham discovered a new kind of attack on block ciphers called differential cryptanalysis. The world then learnt that NSA and IBM knew about this attack since at least 1974, and the algorithm was designed specifically to counter this attack.

In late 90s DES was practically cracked in a contest and then many times after that. The main weakness in DES was the small key size, and to patch it, 3DES was proposed which is still used today, although not recommended. But from an academic point of view, DES is a gold mine. It is easy to understand, let’s us deep dive into the way cryptographers think and learn why certain decisions are made, and most importantly, why the math just works!

DES algorithm from 10,000ft

Okay, let’s start with a super complex diagram that you probably won’t understand without 4 years of formal training in mathematics. Just kidding.


And for the sake of my love for bullet points,

  • DES is a Feistel cipher, which is a family of ciphers which are iterative in nature (repeat a simple set of instructions several times, called ’rounds’) and share many similar properties.
  • DES has a block size of 64 bits, that is, 64 bits of plaintext is converted into 64 bits of ciphertext in one go.
  • The algorithm makes use of a 64 bit key, 56 of which are used by the algorithm and 8 are used for parity check. Effective security is 56 bits.
  • DES has 16 rounds.
  • The encryption and decryption functions are almost similar, which is a great advantage as the implementation and audit has to be done for single function only, simplifying things.

So how does the algorithm work? Like any other algorithm, you can put it down as a list of easy to understand steps.



https://en.wikipedia.org/wiki/File:DES-main-network.png

  1. Take input as plaintext block of 64 bits, and key K
  2. Apply Initial Permutation (IP) on input plaintext (which shuffles the bits in a predefined manner)
  3. Split the input into left half and right half (L0 and R0) (form two equal halves of 32 bits, no tricks)
  4. Apply magic function F (not really) on the right half R0 (32 bits input => 32 bits output)
  5. Function F takes R0 and K1 as input, where R0 is the right halve (32 bit) input for the 1st round and K1 is the 1st round key. In this step, the key material mixes with the plaintext
  6. XOR output of F (32 bits) with L0 (which is already 32 bits), this is the new R1 (L0 ⊕ F(R0) => R1). R0 is simply copied to L1
  7. Thus, we’ve essentially swapped L0 and R0 with some pre-processing on R0. This completes our round 1. Repeat 4-5-6 16 times and you’ve done 16 rounds of DES.
  8. Apply reverse Initial Permutation (a.k.a. Final Permutation or IP-1) and you have your ciphertext. Tadaa!

Yes, I know, that was a mouthful, wasn’t it? This slide [link here] shows the round key Ki in action. Now that we have a basic flow, we can take on each of the components and talk about them in detail, in a proper top down approach.

Little aside on confusion and diffusion

Confusion and diffusion are exactly what they mean in plain English. They provide confusion and diffusion properties in the ciphertext. They are crucial for the overall security of the DES algorithm.

Confusion means having a non-linear, complex relationship between the key and the ciphertext. In simple words, each bit of the ciphertext has to depend on as many bits in the key as possible, such that even with a choosen ciphertext attack scenario, not much can be known about the key given a practically infinite supply of plaintext-ciphertext pairs.

Diffusion means any change in the plaintext should cause an avalanche/snowball effect and change around half of the bits in the ciphertext and vice versa.

We will talk more about how DES achieves both of these properties when we talk about the F function in detail.

DES algorithm: Major parts



Please take a moment to appreciate the effort I’ve put into the diagram. Error: The K(i) should be K(i+1)

We have here the following major components to talk about.

  • Initial permutation, final permutation
  • Round key generator
  • The round function F

Initial & Final Permutation (IP & FP)

The IP accepts the plaintext and the FP returns the ciphertext generated by the algorithm. In decryption, the ciphertext goes into the FP and plaintext leaves through IP, similar but exact opposite of encryption, which is one of the properties of a Feistel cipher. From functionality perspective, it shuffles the 64 bit input block according to a predefined vector, given below.

IP
58    50   42    34    26   18    10    2
60    52   44    36    28   20    12    4
62    54   46    38    30   22    14    6
64    56   48    40    32   24    16    8
57    49   41    33    25   17     9    1
59    51   43    35    27   19    11    3
61    53   45    37    29   21    13    5
63    55   47    39    31   23    15    7

The above text is a linear list, or a vector, and not a matrix. What it says is “take the 58th bit and connect it to output bit 1”, “take the 50th bit and connect it to output bit 2” and so on. It is basically a one-to-one substitution. So how does it, one might ask, help in adding security if the list is public and it is a simple substitution operation. Well, it does not. To quote wikipedia,

IP and FP have no cryptographic significance, but were included in order to facilitate loading blocks in and out of mid-1970s 8-bit based hardware.

Round Key generator



https://www.nagekar.com/wp-content/uploads/2020/12/key_generation.jpg

The round key generator function generates a key for each of the 16 rounds of DES. There are a couple of steps involved, as illustrated in the above visual.

  1. Permuted choice 1 (parity drop) – Get the permuted 56 bit key from the input 64 bit key by dropping the parity bits (bit 8, 16…64 are dropped). The permutation is done according to the predefined vector shown below.
    2. PC-1
57   49    41   33    25    17    9
 1   58    50   42    34    26   18
10    2    59   51    43    35   27
19   11     3   60    52    44   36
63   55    47   39    31    23   15
 7   62    54   46    38    30   22
14    6    61   53    45    37   29
21   13     5   28    20    12    4
  2. Split the 56 bit key into two 28 bit halves, and left shift them either by one bit (for round 1, 2, 9 and 16) or by two bits (for every other round).
  3. Concatenate the two halves thus returned after left shifting, and apply the permutation table 2 to the concatenated pair.
    4. PC-2
 14    17   11    24     1    5
  3    28   15     6    21   10
 23    19   12     4    26    8
 16     7   27    20    13    2
 41    52   31    37    47   55
 30    40   51    45    33   48
 44    49   39    56    34   53
 46    42   50    36    29   32
  4. Permuted choice 2 (compression p-box) – Takes a 56 bit key and returns a 48 bit round key Ki after dropping another 8 bits
  5. The 48 bit round key is then used by our magic function F (remember that?) to mix key into the plaintext by xoring the plaintext with this 48 bit key. (Wait, but our right input halve Ri is 32 bits, right? Yes, we’ll get to how our input is expanded to 48 bits in the next section)

Round Function

We’re finally into the meat of this beautiful algorithm. I’ve mentioned in brief about what the round function consists of. To reiterate,

  • Split the input into left half and right half (Li and Ri) (form two equal halves of 32 bits, no tricks)
  • Apply magic function F on the right half Ri-1 (F takes 32 bits input and gives 32 bits output), where Ri-1 is the right halve of the ith round and Ki is the ith round key. This is where the key material mixes with the plaintext.
  • XOR output of F (32 bits) with Li-1 (which is already 32 bits), this is the new Ri (that is, Li-1 ⊕ F(Ri-1) => Ri). Unaltered Ri-1 is simply copied to Li

What we haven’t talked about is the magic function F itself. The magic function F isn’t really magical. It just does 4 neat sub-operations, and does them really well.



https://www.nagekar.com/wp-content/uploads/2020/12/Data_Encription_Standard_Flow_Diagram.svg

  1. Expansion function E
  2. XOR with round key
  3. S box substitution
  4. Permutation

Let’s look at them one by one and try to see where exactly they fit in and what cryptographic property they give to our ciphertext.

Expansion function
E BIT-SELECTION TABLE
32     1    2     3     4    5
 4     5    6     7     8    9
 8     9   10    11    12   13
12    13   14    15    16   17
16    17   18    19    20   21
20    21   22    23    24   25
24    25   26    27    28   29
28    29   30    31    32    1

As the name might have hinted, the expansion function expands our plaintext input. Expansion gives us diffusion. It diffuses the impact of change of one bit in the input across the block. Remember how the 32 bit Ri part of the 64 bit input is sent to the F function? E function takes those 32 bits of input and expands them to 48 bits. How it does that? Well, repetition, of course. So it basically takes input as 1 2 3 4 5 6 7 8 and outputs something like 1 2 2 3 4 4 5 6 6 7 8 8, effectively increasing the size by 50% (32 => 48).

XOR with round key

XOR is a simple mathematical operation that has a very important property from a cryptographic standpoint. If you XOR a number A with B, you get a new number C. To get A from C, you need B. To get B from C, you need A. Basically, A ⊕ B ⊕ B = A, and A ⊕ B ⊕ A = B. XORing plaintext and key locks them in a interdependent mixture such that to get back the plaintext, you have to have the key with which it was XORed (locked).

S-box substitution

In some ways, this is the heart of the algorithm. S-box substitution gives us confusion. There are eight S-boxes in total, each taking 6 input bits and giving 4 output bits. S-boxes provide DES immunity against differential cryptanalysis which I mentioned at the beginning of this article. Here’s S-box number 1.

      0  1   2  3   4  5   6  7   8  9  10 11  12 13  14 15
-------------------------------------------------------------
  0 | 14  4  13  1   2 15  11  8   3 10   6 12   5  9   0  7
  1 |  0 15   7  4  14  2  13  1  10  6  12 11   9  5   3  8
  2 |  4  1  14  8  13  6   2 11  15 12   9  7   3 10   5  0
  3 | 15 12   8  2   4  9   1  7   5 11   3 14  10  0   6 13

Here’s how it works. After the XOR operation, we are left with a cryptic looking 48 bit string.

say 110010101100101111111100110111101100111010101001

Now we take this 48 bit string and divide it into 8 equal parts of 6 bits each, and input one of the 8 parts into each S box.

SB1(110101) SB2(101100) SB3(101111) SB4(111100) SB5(110111) SB6(101100) SB7(111010) SB8(101001)

Now, our S-box 1 receives 110101.

We take the first and last bit (1 and 1 in this case, coloured yellow), concatenate it to form a two bit number (1 . 1 => Binary(11)) which is 3, and look it up in the row labels of our S-box 1.

Similarly, we take the middle 4 bits (2 to 5), which in our case are 1, 0, 1 and 0, coloured blue, concatenate them to form a 4 bit number (1 . 0 . 1 . 0 => Binary(1010)) which is 10, and look up the corresponding column label in our S-box 1.

The number corresponding to row 3 and column 10 is 3, which is 0010 in 4 bit binary representation. That is the output of S-box 1 for input 110101. Similarly do this for S-box 2-8, for each of the 16 rounds of DES. The result of the 8 S-boxes (4 bits each) is combined to get a 32 bit output.

Permutation

The final step of our magic function F is a simple one-to-one permutation, taking 32 bits and returning 32 bits.

16   7   20  21
29   12  28  17
 1   15  23  26
 5   18  31  10
 2    8  24  14
32   27   3   9
19   13  30   6
22   11   4  25

Catch your breath



I’m really too proud of this picture. Edit: Not so much after finding that K(i) => K(i+1) error.

Wake up! Do you even remember that all this was done on Ri?

Now, after the F function, which wasn’t very magical after all, returns the 32 bit output, we XOR it with Li, which gives us our new Ri+1, while the untouched Ri is simply copied to Li+1‘s place. Hence begins a new round of DES, which goes on this way for 15 more rounds.

After 16 rounds

Not much is left to be done after the 16 rounds. The two halves are concatenated, the 64 bit cipher block is then passed through our final permutation using FP vector given below, and this gives us our ciphertext. Easy.

40     8   48    16    56   24    64   32
39     7   47    15    55   23    63   31
38     6   46    14    54   22    62   30
37     5   45    13    53   21    61   29
36     4   44    12    52   20    60   28
35     3   43    11    51   19    59   27
34     2   42    10    50   18    58   26
33     1   41     9    49   17    57   25

Wrapping DES Up

So that was DES. I hope you enjoyed reading this article. I’m expecting some mistakes, technical and otherwise, so take everything with a pinch of salt. Some interesting reads are given below for those of you who wish to learn more. I realized that writing this article was a nice way of testing my own understanding of the topic, find holes in it and then study to fix those holes. As always, thank you for reading!

Further Reading

Book Review – Applied Cryptography Part I And II – Bruce Schneier

Uncategorized,

This book has been, without a doubt, crucial in aiding my understanding of cryptosystems and why things are the way they are, and how do these cryptic crypto algorithms even work. If you are interested in learning how to develop software that are ‘correct’ and secure, then this is a great book to understand what are the primitives of information security, what algorithms already exist and which ones to use in what scenario.

So the motivation to pursue a thorough understanding of cryptography and to gain the ability and knowledge required to make a secure cryptosystem came sometime after college ended, when I and Kunal were working on a terminal chat application that would support end-to-end encryption. At that time, I hardly knew what I had gotten myself into (which is similar to a lot of things in my life), as the application development part seemed very simple. We got done with the application part, terminal app and the backend, and then came the encryption part, and that is when the knowledge about existing techniques and understanding of basic crypto primitives fell short. And that is when I started reading about cryptography and stumbled upon this book.

Although they seemed daunting at first, both the books are very accommodating for a wide range of audience, right from someone like me who barely knew what a block cipher is, to the more experienced folks who might understand all of the mathematics given in the book in the first go. While not very complex (school grade algebra with addition, multiplication, modulus and xor operations), it takes a little effort (read: re-reading a topic 3 times, sometimes more) to actually get what’s happening, why an operation is being performed, for example.

While reading the first book, remember that it was written when I was literally a year old, in 1996. Hence, although the engineering principles and general recommendation is still valid, you need to keep in mind that the algorithms recommended in that book are not valid (as attacks are found for many of them and DES has officially retired), and that is corrected in the second edition of the book. In any case, studying the DES algorithm in detail should be a delight for any crypto nerd, regardless of its practical value.

The second version is more up to date, and for some reason I was more comfortable reading it than the first one. It might be because I knew a little more while reading the second edition, which can be a good tip: If you’re serious about understanding cryptography from an engineering standpoint, skim over the first book and make a note of everything that you find useful and interesting, and do a more detailed study of the second edition of the book.

What I found nice about the books is, they really are ‘applied’ books. It isn’t all mathematics and algorithms, but the actual merger of these algorithms into real world systems. In the real world, cryptography and cryptosystems don’t exist in isolation, but play a small role in the larger scheme of things. Breaking a cryptosystem is usually reserved for the more resourceful adversary, and while these (well established and peer reviewed) cryptographic primitives rarely fail, when they do, it is catastrophic. The computational infeasibility makes the theoretical aspect of cryptography very secure. Problems appear when they are implemented, and that is where the bugs start to show up. Then there is the software development methodology which usually prioritises deadlines and features above security. There is a section dedicated to explaining what ‘trust’ is, how it forms such an important aspect of information security and secure software development. Overall, the book is quite interesting to read, and the content is without a doubt top quality, which is what one expects from Schneier.

In closing, I’d recommend this book if you are into security and wouldn’t mind knowing the details of some of the fundamental algorithms that make the digital revolution possible. Thank you for reading.

Book Review – Responsive Web Design By Ethan Marcotte

Uncategorized,

It has been a while since my last book review post here. Not that I stopped reading, but I kinda stopped reading non-tech things lately, and hence, there were no new posts. But today, it hit me that I can (and should, given this is a personal diary) write about pretty much anything that I read and find interesting. So here it is, a book by Ethan Marcotte, which I first read about a year and a half ago and then re-read it before a month or so. Responsive web design wasn’t (and still isn’t) my piece of cake. Heck, web design was something totally alien to me in the first place.

The happy realization that being able to set up websites (read: wordpress/joomla blogs on a nix server) doesn’t make one a web developer, much less a designer, came about two years ago, when Dhananjay, a college senior of mine, was contacted by one of his contacts who was looking for a frontend developer. The task was supposed to take a couple of hours at max. Knowing that I did things around the web, Dhananjay outsourced that opportunity to me.

That was one incident that still gives me chills, and I wrote a bit on that earlier. Not only because I realized how horrible I was with frontend and design, but also because I didn’t have the slightest clue about deadlines, how to and how much to work, and how to deal with things that are out of my control. It was a design heavy page, and I had a depth first approach of dealing with things. The end result was that a few pieces took up 80% of my 5 days of work (easily worked for over 70 hours), and the end result was nothing short of a design disaster. That one incident has taught me a lot, especially about how real work happens.

I guess it was then when I had read Ethan’s book for the first time. I believe it wasn’t as much for learning as it was to put on some burnol on my bruised ego. But nevertheless, even then the book had given me much insights about what web designing actually is, and why it isn’t very different from what I had been doing all along, it just requires thinking in a different mindset.

Fast forward to June this year, I interviewed at a couple of places for the role of a web developer. I was expecting a role on the backend, maybe a nodejs or python based job, but instead, I got a job as a ReactJS engineer. Yeah, a frontend engineer. As difficult as it was for me to digest it, I had to accept the fact that I will be dealing with a lot of CSS now. I had to up my design game, or it was game over, and I seriously didn’t want to screw as bad as I did two year ago. My friend Kunal was kind enough to lend me his Head First HTML & CSS book which I am currently reading. But apart from the raw knowledge, it was the mindset that I required immediately, the mindset of a frontend developer, and for that, I picked up Responsive Web Design once again.

Shall we start with the review, Plis?

Sure. The author starts by talking about architecture, responsive architecture in particular, about artists and their canvases. Responsive architecture is all around us, from window panes that allow variable amounts of light depending upon the time of the day, to modern home equipments. The author then talks about the usual restrictions in print media, and how web designers are fighting hard to recreate those restrictions on our browsers. We do not have to do that. The canvas of a web designer is inherently responsive. It isn’t a flaw, it is a freedom.

The author makes sure that reading this book won’t feel like the usual wall-of-text-hitting-your-face-with-technical-jargon experience. The book feels like a spiritual exercise, as if web designing is an art waiting to be discovered by an engineer who always saw it like a soul dead practice of giving random attributes to random elements and praying to the Gods of DOM that it looks just decent enough to pass the QA. I was really immersed into the book as I was reading it, and hoping that it lasts forever, which it obviously didn’t. The book is not long, and is divided into three sections; The responsive grid, Responsive images and Media queries. After reading this book, you’ll look at hardcoded ‘px’ values as if they were taboo in your (code) culture. The author shows how simple calculations can turn all the zombie pixel measurements into the more lively ’em’s and ‘rem’s, which are, of course, responsive.

A good article that the author strongly recommends is a blog post that was written some 17 years ago from now, but still is as relevant today as it was then. The post is called A Dao of Web Design, and it falls into the must-reads category for me. To give you a taste of the article, read the following quote.

The control which designers know in the print medium, and often desire in the web medium, is simply a function of the limitation of the printed page. We should embrace the fact that the web doesn’t have the same constraints, and design for this flexibility. But first, we must “accept the ebb and flow of things.“

Beautiful, isn’t it? Suddenly, web design isn’t something that you do when you’ve done everything you could do to avoid it in the first place. True, writing CSS by hand is time consuming, working and supporting multiple browsers and display sizes is stressful to say the least, and most of the time, you’re better off using a ready-made solution like Bootstrap or Semantic, but once in a while, it is good to think about web as your canvas and think of yourself as an artist trying to fill in beautiful colors into the canvas. Now whenever I think about the different ways in which my web application is supposed to look on different screens, I remind myself that it isn’t a ‘restriction’ that the app needs to look that way on that screen. Rather, it is a freedom that the app can look the way it needs to look in order to be the most comfortable version of itself for that particular reader. Ever seen a person struggling with folding a newspaper on a busy bus stop, or a cautious women carrying a large piece of art in her arms, making sure she doesn’t bend it, yes, that is exactly what a restriction, a limitation looks like. Thankfully, our dearest web doesn’t have that one. Thank you for reading.

Tinkering With OBD-II Port

Uncategorized,

I’ve been seeing people hook up their computers to their cars from quite some time. It is a common sight if you watch any motorsport event on television, where technicians are seen working on their laptops that is connected via a cable to the car or bike. I found it quite fascinating. “What interesting tweaks must they be making to that machine with that computer!” I thought. The idea of tweaking a machine to improve it’s characteristics wasn’t new to me. Overclocking is nothing new. But obviously, since I saw all those professionals do it, I assumed there was no way for such an interface to exist on our everyday road vehicles.

And I was wrong. I discovered that, by law, it was necessary for all cars to have a diagnostics port, called the On-Board Diagnostics port. The latest revision for that port is v2 or OBD-II, and all cars manufactured after 1996 should have one. Also, sometimes, the automotive Youtubers I followed showed various stats on the screens such a the engine rpm, throttle position, boost pressure etc. So that implied there exists a way to extract those stats out of the vehicle’s ECU. Interesting. A quick Google search for “odb scanners” revealed that they’re not very expensive either (with cheap clones available for as low as INR 300, USD 5 or even lower). After researching a bit, I learned that there was loads of data that came out of that little adapter, and that great Android applications (like Torque and DashCommand) exist which spit out the data into beautiful dials and graphs (like the ones on the Nissan GTR ♥) I was awestruck. What more can a nerd ask for!

All this happened a couple of months ago. I knew I needed to get one of those. I waited a couple of months and finally ordered it earlier this month. The first challenge was to find the OBD port. Unlike some other cars, Zacky’s OBD port was hidden behind the fuse box cover, the adapter had to go inside there. I managed to access the port without opening the fuse box and problem solved! Plugged in the adapter, paired with with my phone and it started sending data. That was one of the best feelings ever!

Some of the data it sent that I found particularly interesting to read was

  1. Boost pressure from the turbocharger
  2. Engine RPM
  3. Coolant temperature
  4. Engine load
  5. Error codes and provision to reset them
  6. Horse power, torque, acceleration and other such “calculated” data by combining sensor data with phone’s sensors like GPS and accelerometer and known parameters (like vehicle weight, engine displacement etc)
  7. and loads of other cool stuff

Note that the available sensor list varies from manufacturer to manufacturer, so keep that in mind. But even with the most basic, the experience is fun. It’s like opening task manager on your computer for the first time. Wow, so I can actually run this h4ck3r stuff, right?

Interesting Learnings

– Negative boost pressure When you start the car and drive it normally, you’ll notice that the boost pressure gauge will read negative (technically, not pressure but vacuum). Only when driving hard (shifting late, for example), will you notice the boost pressure rising. I thought it was some erroneous data from the sensor so I read up a bit. Turns out, at high rpm, the turbo forces the air fuel mixture into the cylinders. But what happens when the turbo is running too slow for compressing air? It simply works as a naturally aspirated engine and sucks in air during the intake stroke. THAT sucking part explains the vacuum. Cool!

– Driving modes So Zacky featured this thing called driving modes. Putting her on “Sports” made the throttle more responsive but reduced fuel economy while putting her in “Eco” did the exact opposite. Now I could’ve told you that this isn’t just marketing and if you test it out, you can even feel a noticeable difference, but that was all I knew. Now, after driving for a while with the boost pressure gauge in front, I made this little observation. When in normal drive mode, the turbo does not spool over 4-6psi boost. But as soon as I go ‘sport’, the turbo goes well over 10psi, even 12 if the sensor is to be believed, which is pretty fantastic.

– A better understanding of the relationship between torque and horsepower, and what each number actually implies. Yes, power is work done per unit time, but what exactly does that feel like. Why do diesels have same horsepower figures even after having loads of torque. It gets really clear once you see the torque, the rpm and the (thus calculated) horsepower figures side-by-side.

– Torque curve So there’s this thing called a torque curve of an engine, which is just a curve with torque on one axis and RPM on the other. For an IC engine, the torque is not linear (as with electric motors), but a curve with a peak at some specific RPM (or RPM range, which is why a torque (or horsepower) figure is always accompanied by a RPM range), and tapering off at both the ends. To get the maximum acceleration, you have to keep this curve in mind when changing gears.

Now show me some kode!

Yeah, right. So while I was on all of that, I thought, why not study the protocol itself and try writing a little script to pull the raw data from the sensors out, just for fun. Right, but how? This thing is running on Bluetooth, and how do you sniff that. Is there something like Wireshark for bluetooth? Googling “Wireshark for bluetooth” reveals that Wireshark is the “Wireshark for bluetooth”. Damn!

But before wireshark could sniff, I needed to get thing thing connected to my laptop. That’s pretty straightforward. After having it running at /dev/rfcomm0, fire up Wireshark and keep it listening on Bluetooth interface.

Okay, pause. Here’s the funny part. The above text was written some 4 months ago. Then I had to do a lot of physical work to take my laptop into Zacky and do all the research/coding from there. I remember going out at least 3 times, but for some weird reason, never bothered to finish writing this article. I’m putting this out right now so that I will remember to write the part-II for it during the next weekend. Stay tuned.

Better Web Browsing

Uncategorized,

My web browser is by far the most threat-exposed application that I use on my laptop and mobile phone. Not only does it have to trust web developers and run the scripts they wrote, which probably aren’t all that great with respect to user security and privacy, it also has to deal with literally thousands of ad networks and other third party scripts that are specifically designed to invade your privacy by tracking you online.

The people behind these nasty things are not crooks or cyber criminals working from a shady basement, but are very smart people, working for the top Internet companies and making themselves and their employers more money than what you and I can imagine in our humble minds.

Given that billions of dollars are at stake, you and me as common Internet users don’t have much of a say in this. They will track you, trade your data, make money while you enjoy their ‘free’ services, and kick you out the moment you don’t comply with their policies or stop agreeing to their rules.

The only thing that we control is what data are we giving to these Internet giants. While not many people realize or even care about it, there are ways in which we can limit our data leaking to the Internet, and the first step in doing that is hardening your browser. This article is an attempt to cover a number of steps everyday Internet users can take to protect their identity online and improve their information security. I was inspired to take up this topic for writing after a detailed explanation on the same in Nathan House’s Network Security course on Udemy. Do check it out.

Know thy browser

Your browser is a very powerful piece of software. The level to which it can be fine tuned to serve you the content that you wish to see is huge. The about:config page has some very useful options for both privacy and security. While almost anybody might have a hard time configuring Firefox via the this menu, there’s a helpful website called ffprofile.com that does a great job at making sense of some of those options and exporting a pref.js file.

Make sure you read the options and select the ones that you need depending on your privacy and security needs. For example, I deselected the disable auto updates option because the privacy gain from doing it is minimal (for me), while the security trade-off is huge, potentially leaving me unarmed at times. If you, like me and most others, can’t go full Stallman-mode anytime soon, make sure you set up an alternate profile in Firefox (firefox -no-remote -ProfileManager) and set up non-secure browsing preferences there, selecting it via firefox -no-remote -P username, or using an addon. Learn more about Firefox profiling here.

Apart from that, try to use a non-tracking search engine like DuckDuckGo, StartPage etc, use incognito mode by default (in Preferences > Privacy and Security), and use history saving only when required and on trusted sites. Use HTTPS whenever possible (Check out EFF’s HTTPS Everywhere addon). There are addons available to notify you of certificate change in HTTPS enabled sites (which can hint a MITM attack), but they are of little practical value as sites update their certificates all the time. There are a bunch of useful addons that will come bundled with ffprofile, but you can also download them manually from Mozilla’s addon store.

Disable 3rd party cookies

3rd party cookies are used for tracking users on third party sites. It is usually harmless to disable them. What might not work after disabling them are your social like/share buttons and commenting platforms like Disqus, but that’s what we want in many cases. You can also consider setting up delete all cookies when you close the Firefox window.

Use a VPN

VPNs are not really part of browser hardening checklist, but they offer good privacy over any insecure network. A lot of middle men in your Internet packet’s journey can potentially know what sites you visit, especially on insecure HTTP connections. Even on HTTPS, your browser’s DNS lookup might give away some information about your browsing habits. On insecure HTTP website and a shared Wifi access point, you can assume at all times that the other users connected to that Wifi, the access point and the ISP can literally see each and every request that you make. VPN takes away this data leak by creating a virtual tunnel between your computer or mobile device and the VPN’s server. The connection is encrypted and hence sniffing cannot happen in between. All requests, even DNS can be (and should be) configured to use the VPN.

It is important to understand that sniffing can occur on the VPN’s end on an insecure connection, and hence you need to select a VPN provider with utmost care. Even after this, there’s a bit of trust involved when choosing a provider. Our best bet is to try to opt for a provider that maintains a zero knowledge service.

Use a password manager

Although it is an extremely bad practice to write passwords down, another very common mistake we as Internet users do is reuse passwords on many sites (I’m guilty as well), which in some cases is worse than writing down passwords for each individual online account. We know that at least some of the sites store your password in plaintext, while a lot more use weak hashing algorithms. Since we can never be sure, always assume that the password you submit to any site can be accessed by an adversary, and used against you. If you’ve reused your email provider’s password on any other site, the website’s admins or any attacker who has/gets access to the website’s database might be able to take over your email account and other services connected to that account. It is for this reason that using separate passwords become important.

However, as human beings, we have more important things to remember than random alphanumeric strings. This is where a password manager comes in. It takes away your responsibility of having to remember 15 different passwords by making you remember one master password. Sweet, huh? This might look like an extra point for failure, but in the broad scheme of things where an attacker might get one of your password and literally own your digital life, this is a much safer option. What’s more, the passwords are either never stored on the disk (generated on the fly based on the master password and website’s name) or stored in encrypted form on the disk (or in the cloud). On the downside, this also means that if you lose your master key, you lose all your accounts.

Use 2 factor authentication. Always.

Whenever it is an option, use a second factor of authentication. It can be anything; a phone call, SMS, email, authenticator app (like freeOTP) or whatever. The increased effort is totally worth it considering the alternative is to risk an asset, possibly an important one. Remember that defense in depth is an effective concept in security, both in the real and digital world, and we must make use of it wherever possible.

For the paranoids amongst us: Use a live operating system

A live operating system does not store any data in between sessions. Every time you log into the system, it is like logging into a freshly installed operating system. Naturally, there’s not much tracking that can be done, and as nothing is ever written to the disk, this method offers best privacy if done properly.

Using applications in virtual machines also protect users against sandbox escape vulnerabilities. Since we’re here, check out Tails Linux, a gnu+linux distribution that is designed to be used live and offers great tools to aid privacy. Another great live operating system is Whonix, which comes in form of a ‘gateway’ (connects to the tor network) and a ‘workstation’ (connects to the gateway). Then depending on your hardware, Qubes OS might be a good choice, something that I look forward to trying when I have compatible hardware.

Lastly, remember what Bruce Schneier told us

“The question to ask when you look at security is not whether this makes us safer, but whether it’s worth the trade-off.”

I’ll leave you with that TED talk here. Thank you for reading.

Mozilla Firefox Is Back!

Uncategorized,

It is no secret that I’m a huge fan of Mozilla, the organization, and their browser, Firefox. The reasons are pretty clear, it’s a decent browser, comes with tonnes of customization capabilities, is backed by an organization who’s business model doesn’t involve knowing each little detail about you as a person, and is made and backed by the free software community.

I used the word decent, and not great or amazing, because it is not. It isn’t as fast as Chromium, feels sluggish and looks outdated. The other advantages still remain, but the impatient Internet citizen of 2017 isn’t going to take a slower browser for idealogical reasons. And I’m feeling extremely proud to tell you this is exactly what Mozilla got right this time. Firefox 57 isn’t just a little cosmetic update to the previous build (although I would’ve even celebrated a cosmetic update), it is like the first major upgrade Firefox received in years. And guess what, it is fast. Very fast. Chromium fast.

I’ve started using the beta (the stable should come out in November), and it feels pretty solid. The UI is more fluidic, rounded corners in tabs are replaced by more modern rectangular ones, the preferences page is all changed and so have most things. If you’re a long time Firefox user, this update is going to feel like a dream. If you’re a Chrome user, then well, you’ll feel some similarities, right at home, but now you have the usual power of a free software. You can see a comparison video posted by Firefox a few days ago below.

If this interests you, find out more about Firefox Quantum by clicking here. I really liked it, and I’m sure you will too. Just give it a try. Download it via this page that lets you select a build for you operating system and locale.

Private Cloud Part 2 | Encrypted Storage With NextCloud

Uncategorized, ,

New cloud setup. YAAY! Self hosted, encrypted and scalable. Plus comes with a nice web interface, native Linux and Android clients and its very own app store. I’ll first write about the setup itself, and then some of my personal thoughts over the entire private cloud exercise.

Features Overview

The major components of the setup include the following

  • NextCloud 11 on Ubuntu using Digital Ocean’s one click installer on a 5 USD cloud vps
  • Digital Ocean’s flexible block storage
  • Let’s Encrypt for free TLS
  • NextCloud sync client for Arch and Android on desktop and phone respectively for data sync
  • DavDroid for contacts and calender sync on Android (uses WebDAV)
  • Optional redundant backup and client side encryption using GnuPG (see below)

Pros Vs Cons

So I now have a proper private cloud, self hosted, synced across mobile and desktop (including contacts, messages and calender), optional client-side encryption and scalable (♥DigitalOcean♥). What’s amazing is that I never had a native Google Drive client on desktop, but now I have a native NextCloud client, and it just works. And yes, it isn’t all sunshine and rainbow. There are some serious trade-offs which I should mention at this point, to make this fair.

  • No Google Peering, hence backing up media is going to be a struggle on slow connections
  • Google’s cloud is without a doubt more securely managed and reliable than my vps.
  • Integration with Android is not as seamless as it was with Google apps, sync is almost always delayed (By 10 minutes. Yes, I’m an impatient (read ‘spoiled’) Google user)
  • Server maintenance is now my responsibility. Not a huge deal, but just something to keep in mind

Having said that, most of it is just a matter of getting familiar with the new set of tools in the arsenal. I’ve tried to keep most things minimal. Using few widely adopted technologies and keeping them regularly updated, sticking to the best practices and disabling any unwanted, potentially dangerous defaults and with that the server is secure from most adversaries. Let’s first define what “secure” means in the current context using a threat model.

Threat Model

The only thing worse than no security, is a false sense of security

Instead of securing everything in an ad hoc fashion, I’m using this explicitly defined threat model, which will help me prioritize what assets to secure and the degree of security, and more importantly, what threats I’m NOT secure against.

  • Compromised end device (Laptop): Since data is present unencrypted on my end, an adversary having access to my computer via say a ssh backdoor can easily get access to all of my (unencrypted) data. Private keys cannot be compromised as they are password protected. A keylogger might be able to sniff out my password which can then be used to decrypt any encrypted data.
  • Compromised end device (Mobile phone): Since data cannot be decrypted on the mobile, all encrypted data would remain secure. Only the unencrypted files will get compromised. However, if an adversary gets access to my unlocked cell phone, securing cloud data would be the least of my worries.
  • Man In The Middle (MITM): As long as Let’s Encrypt does it’s job, TLS used should be enough to secure the data against most adversaries eavesdropping on my network. It would not protect me if Let’s Encrypt (or any other CA) gets compromised and an adversary makes duplicate certificates against my domain and uses it to eavesdrop the traffic, the possibility of which is rare.
  • Server Compromise: If the server is compromised through any server side vulnerability (assume root access) and an attacker gets access to everything on the server, all unencrypted files are compromised, which would include contacts/calender lists. Since the decryption key is never transmitted to the server, encrypted files won’t be compromised.

Why Client Side Encryption

The entire exercise would look pretty pointless if I just took all my data from G Drive and pushed it to NextCloud. And from the previous cloud server attempt, I know how uncomfortable it is to have your data accessible from the network all the time. Those reasons were more than enough for me to go for an encrypted cloud solution. Although it would still look pointless if you were to ask me why didn’t I just encrypt the data and upload it to G Drive again. The answer is simply because I didn’t want to.

After some research (being a novice with security, that was a must), I came up with a list of guidelines that I had to write my solution on.

  • Use of symmetric key cryptography for file encryption, particularly AES-128
  • Memorizing the AES key or using public key cryptography to store the key of file en/decryption on disk. (Not sure which is the proper way of doing it, although I’ve asked the experts for help)

Encryption

There are a lot of tools one can use for data encryption. I used Gnu’s Privacy Guard (GnuPG or simply GPG). It is anything but easy to use. But the nice part is that it just works, is extensively reviewed by experts and has been around since I was 4 years old. So in theory,

  • Generate a public/private key pair in GPG
  • Generate a strong passphrase for the encryption, and encrypt it using the public key you just generated. Store it locally someplace secure
  • Get a list of all files and directories from a specific folder using find (for one time backups), or use rsync with a local sync copy (for incremental backups)
  • Iterate the list (of all or changed files). If item is a directory, create that directory, if item is a file, encrypt the file and push it to that directory.
  • After encryption, you’re left with either two or three directories, /original-dir, /remote-encrypted and optionally, /local-unencrypted-sync
  • The additional (local sync) directory is useful when incremental backups are required and rsync uses this directory to keep track of changes, and only (re)encrypts those files that have been added/changed since last sync. Useful to setup a cron job. At this point, you can delete the files in your /original-dir safely
  • Decryption is just the opposite of this. You supply the location of your /remote-encrypted directory and the script generates a new directory with unencrypted content.


Original directory


Encrypted backup directory

This does the job for now. Here’s the script that I’m currently using. I wanted to enable sync without the need for a helper directory, just like Git does (it stores the changes in the same directory in a .git/ directory). Will update it if I manage to get that done.

In Closing

Eighteen months ago, I wrote on how to create a ‘cloud’ storage solution with the Raspberry Pi and half a terabyte hard disk that I had with me. Although it worked well (now that I think about it, it wasn’t really a cloud. Just storage attached to a computer accessible over the network. Wait, isn’t that a cloud? Damn these terms.), I was reluctant to keep my primary backup disk connected to the network all the time, powered by the tiny Pi, and hence I didn’t use it as much I had expected. So what I did then was what any sane person would’ve anyway done in the first place, connect the disk with a usb cable to the computer for file transfers and backups.

Earlier this year, I switched ISPs and got this new thing called Google Peering, which enabled me to efficiently backup all my data to the real ‘cloud’ (Google Drive). That worked, and it was effortless and maintenance free. And although Google doesn’t have a native Linux client yet, the web client was good enough for most things.

And that was the hardest thing to let go. Sync and automatic backups were, for me, the most useful feature of having Google around. And while everything else was easy to replace, the convenience of Drive is something that I’m still looking for in other open source solutions, something I even mentioned in my previous post on privacy.

So although I now have this good enough cloud solution, it definitely isn’t for everyone. The logical solution for most people (and me) would be to encrypt the data and back it up to Google Drive, Dropbox or others. I haven’t tried, but Mega.nz gives 50GB of free tier end to end encrypted storage. Ultimately, it makes much more sense to use a third party provider than doing it all yourself, but then again, where’s the fun in that! Thank you for reading.

Privacy – How I Converted

Uncategorized

In spite of my inclination towards cyber security from an early age (relative to when I ‘discovered’ the Internet), I never was a big fan of privacy over the web. I knew some bits here and there about it, like how my data is used to serve me targeted content, how tracking happens even after I close the browser tabs and how companies watch me visiting sites and track my habits. Heck, I found it fascinating that I saw adverts from third party companies about the products that I was currently researching about. Internet, to me, was like a close friend who knows everything about you, your habits and interests, your lifestyle and more. And when I say friend, it isn’t metaphorical. I literally trusted the web for every bit of work-life thing that I got involved into. I liked that my email was always synced, that Google asked if I wanted to review the place I was at, that all my photos were automatically backed up to the cloud, that I got a ‘This day 3 years ago’ notifications every once in a while, that I received personalized notifications about the bills that were unpaid and the events that were due, like magic!

And all these years, I’ve heard about numerous leaks, activists exposing unethical government secrets and mass surveillance and I was always disconnected from it. When Airtel & BSNL were injecting adverts into my web pages, I was okay with it. When Google or Whatsapp changed their privacy policies, I readily accepted the new ones, after all I’m sure they value their users, and decide in their best interests, right? After all, what do I have to hide?

Now, I consider myself a huge fan of free and open source software, and in the open source world, you readily trust the software or content, not because you personally trust the people behind it, but because the code is subject to scrutiny by fellow community members and as a result, the chances of using an open source software that is a malware or a back-doored Trojan is essentially zero (such attempts are readily caught. (meta meta: is this a survivor bias?)). I remember the heavy criticism of Ubuntu for logging the search keywords of it’s users for serving them targeted ads which eventually led to elite members of the open source community advising against using Ubuntu and RMS calling it a ‘spyware’. But what Ubuntu did is only tiny bit as harmful (they did put an option to opt out of this ‘shopping lens’, or uninstall it altogether) as some of the tools and services we use everyday. And that is what I realized in the past month.

From here, it is about how I turned 180 degrees and started to care about privacy and anonymity more than ever, how I became paranoid about the data that I publish online and think twice before registering for an online service, or visiting untrusted websites without a VPN. If you feel this is of no interest to you, I urge you to close this tab after watching the following video. The message is very powerful and I’d like you to give yourself sufficient exposure to the problem before deciding if you want to care. You may continue reading if you would like to learn about my decision and what led me to it.

Let’s start with the most obvious question…

Why now?

The anticlimactic answer is, better late than never. This article isn’t the result of a single blog post that I read or any specific incident. It is a cumulative result of the critical exposure I’ve had in the past month or two, and a subconscious exposure of the past few years. I had this on my mind from some time, but laziness is what I’d call it. Who wants to give away the convenience of synced devices and automatic backups! I’m fortunate enough to have a paranoid friend around who doesn’t use many (any?) social networking sites and online services. All he has is probably a ProtonMail email address, and he’s just as active on the Internet as I am. I always considered his view of privacy a personal preference, a subjective view of the world, not an objective truth about the Internet and companies based on Internet. But recently, the more exposure I’m getting about the way Internet giants collect and use my information, government surveillance etc, the more I’m moving away from using their services. It isn’t about if someone is watching me while I use the Internet, which no one probably is, given my uninteresting Internet activities. It is the possibility that at any given moment someone/something could watch me, without my consent, store tonnes of meta data about me for use 15 years from now, and I might lose the basic right to privacy that I always took for granted, is what makes me uncomfortable.

However I don’t expect anything to change when I make a switch. In most cases, nothing would change for me, as an individual who accesses and relies on the Internet everyday. Free and open source alternatives exist and it is a matter of hours (if not days) to make a complete switch from proprietary to open source software. But now, I’m leaving a lot less footprints in random server logs and by using open source whenever possible, I can narrow down the number of malwares and spywares I carry around with me in my phone or laptop. And something I really need to emphasis on, a spyware is not necessarily installed by just a third party malicious user. OEMs ship spywares all the time (tampering preinstalled TLS certificates and performing MITM attack to show ads, now that’s dark). All this is without even mentioning the humongous quantities of crapware these OEMs ship their products with, widening the attack surface for a third party adversary. All of this can be mitigated if you control what’s installed on your devices and choose what services to use.

If you want to know more about what sort of threats to privacy exist around you, you might want to check out this amazing course by Nathan House titled ‘The Cyber Security Course – Hackers Exposed’. Don’t get intimidated by the title, it is for anyone who wishes to understand the threat landscape so that he can take the necessary steps to ensure adequate security and privacy according to his needs. Nathan does a great job at putting the key points in front for you to decide rather than feeding you his opinions. Highly recommend his course.

Is the threat real?

This question arises in the minds of people when they hear about issues like Privacy and Global Warming (I was surprised to find a good number people think Global Warming isn’t real). Is this real? Or is it one of those hyped-stories that would fade away and everything will get back to normal once media stops covering it. Let me start by confessing that it was in the last month that I read the terms of service of any company I used online for the first time, and boy I was surprised. I agree that reading ToS is boring, but it really is critical to ensure a peace of mind when you use a service. If you’re still not sold, check out this amazing site called tosdr.org or Terms-of-service-didn’t-read which summarizes the ToS of popular services and rates them from class A (good policy) to class E (bad policy) and the key reasons supporting the rating. The data is a bit outdated, but you do get a general sense of the corporation’s privacy structure. And to be honest, you don’t need any of this. All you need to do is keep your eyes and ears open and assess the data you’re about to give to the next application you download from the market. Take Whatapp’s ToS for example, the service which promises that the messages are end to end encrypted with Signal Protocol. Sure, they are. And there’s no doubt in my mind that Whatsapp is one of the most secure messengers we have with us today. But privacy and security are two very different topics to discuss, both equally important (a good read here). And when it comes to privacy, it is not our messages or content that companies usually target. It is our meta-data. Here’s what Snowden tweeted about it.

Are your readers having trouble understanding the term "metadata"? Replace it with "activity records." That's what they are. #clarity

— Edward Snowden (@Snowden) November 2, 2015

Now there are a lot of articles on this topic and hence I don’t plan to get into it. To quote a key point from one of the articles about what meta data really is,

  • They know you rang a phone sex service at 2:24 am and spoke for 18 minutes. But they don’t know what you talked about.
  • They know you called the suicide prevention hotline from the Golden Gate Bridge. But the topic of the call remains a secret.
  • They know you spoke with an HIV testing service, then your doctor, then your health insurance company in the same hour. But they don’t know what was discussed.

This example should not imply Whatsapp is the only or worst offender, it is just one of them that I’m familiar with (and use personally). I assume that you can read and decide for yourself.

Why did I care?

“But Government Surveillance is a US problem!”, “My nation doesn’t come under 5, 9 or 14 eyes“. Sure, but don’t we all use the same Internet? I don’t need to emphasis about the importance of Internet in matters of freedom of speech, how it has nullified international borders to connect people of similar interests and how many revolutions it has started. I don’t need to mention that Internet is a second home to political (h)activists and dissidents, a place where they can express themselves to the masses. I certainly don’t need to mention what I personally feel about the Internet which you probably know by now. And I’m not even getting started on how, even if you don’t belong to any of these ‘eye’ nations, most of your traffic is still getting routed through them.

The way I think about it is, a full disclosure sometimes becomes necessary to bring about a change in the system that is broken and is resisting a fix. This was one of the highlights in Keren Elazari’s TED talk on ‘Hackers: Internet’s immune system’

Precautionary measures I took

I adopted some defensive measures to try out this new Internet lifestyle, applying the learnings from the past couple of months, since it wouldn’t make sense to not do it after this exposure. This is quite experimental, so try out what works for you, the way I’m doing. A word of caution though. This list would not cover you from threats and privacy breaches from third party adversaries like cyber criminals, who might choose targets more specifically (like sending a malware via email or infecting your local network). The best (and in many cases, only) defense against it is to keep your systems (laptops, mobile phones) up to date with the latest security patches. Did I make it sound important enough? KEEPING YOUR SYSTEM SOFTWARE UP TO DATE AND PATCHED IS THE BEST THING YOU CAN DO TO STAY SECURE. Sorry for screaming. Okay, now back to the measures I took.

  1. Flashed LineageOS on my Phone – Almost stock Android, plus more control over what I install (note: rooting, flashing, installing from unknown sources etc potentially opens a huge security hole in itself)
  2. No Google Play Services – The suite of Google apps such as Gmail, Youtube, Docs and Drive are optional, and I chose to not install them
  3. Gave up my G Suite subscription. So no synced devices and automatic photo backups. (Remember to ‘Takeout’ data before leaving)
  4. Turned off port forwarding, DMZ, UPnP and any other service on my router that might expose any of my internal devices to the Internet
  5. K-9 Mail as email client
  6. SkyTube for read-only tracking-free Youtube
  7. f-droid for free and open source Android apps, also there are plenty of closed source apk repositories that don’t require a Google account.
  8. DuckDuckGo as the default search engine across all devices
  9. Debian or Arch linux on desktop, as recommended by Nathan House, provides a good mix of active development, security, support and speed, although you can pretty much choose any good distro depending on your taste and harden it.
  10. Signal Messenger on Android/iOS for Whatsapp like security and usability minus the meta data issues
  11. Firefox Focus as the primary browser on phone, except when explicitly wanting to store history, in which case Mozilla Firefox
  12. Mozilla Firefox on desktop, and Chromium as the secondary browser. Google Chrome is a better browser imho, for it supports a lot more content types than Chromium does out of the box. Not to mention better updates and security. Boils down to your personal preference, really
  13. Deluge for torrents
  14. LibreOffice for document/presentation editing needs
  15. VLC for pretty much everything multimedia
  16. And the rest of the goodies you get with any nice distro. (Must admit that I haven’t found a Google Drive alternative yet)
  17. Lastly (and optionally), encrypted mail providers like ProtonMail for secure email and a good VPN (such as Mullvad, or Tor for that matter, but make sure you read the differences) for use when on public Wifi hotspots

Needless to say, that is what I’m using right now, and kind of recommend. Except for the couple of options on the top, I’m sure most of you are familiar with (and probably use) the rest. If yes, that awesome. That’s a win for the free and open source community. And I’m not affiliated to any of those! Haha

Is any of this necessary?

“You are overdoing it!” as my friend exclaimed. I totally agree, and to be really honest, it is not just about privacy at this point, it is about enjoying the new world that I’ve found, exploring the corners and trying to fit in. I believe that open source shouldn’t feel like a compromise. It should be a pleasant experience for everyone who uses it, whether or not they consciously care about it being free and open. I am sure not everyone is so willing to give away convenience for the sake of some principles and ‘freedom of the web’, and that is totally fine. As long as you take the decision of giving away your data and are okay with it for the rewards it comes with, and not let a corporation decide it for you, I’m no one to tell otherwise. I’m here to tell you that there’s a world out there that represents the open and free nature of the Internet, and it is not at all difficult to convert. I did, and so can anyone.

Links from the post

Aggregating all the blog / additional information post links with their titles from the above text here.

Thank you for reading.

A Programmer Or A Problem Solver?

Uncategorized,

Normally when we think of programming, we think of problem solving. Similarly, if someone works in the field of computers and say they love problem solving, we immediately assume that they work with some computer code. Programming is almost synonymous with problem solving, in that it involves breaking down complex looking problems into simple mini problems that can be easily taken care of. Where they might differ, in my opinion, is finding the right problems to solve.

The inspiration for this post came from a recent blog post that I read (embarrassed to admit I’m not able to find that post found it!) about the mistakes developers make. One of the mistakes was confusing between the love for problem solving and programming. That was a little “ahaa!” moment for me. It gave me a moment to reflect on my own likes and interests. What is it that excites me? Is it the idea of building the next big thing? Maybe. Is it spending countless hours writing code that does what has been done a million times before, just so that you can fall in love with your code all over again? Yes, that’s sounds about right.

I liked to call myself a problem solver, but I’m not even close to being one. I didn’t feel like there was a distinction. But there definitely is, now that I’ve met some people in my field who are ‘problem solvers’ first. I don’t have Github projects that reflect a problem solver. What you’d rather find are spot on examples of reinventing the wheel (a dozen chat/social network networks), attempts to write the most beautiful code that I can (regardless of whether it works or not), over-engineering to say the least, projects made entirely for trying out new languages, new frameworks, new IDEs, literally. I’m somewhat embarrassed to admit that the current project I’m working on is a chat backend as well.

You get the pattern. It isn’t hard to understand that I love programming. I love writing code regardless of the problem in hand. I see people building things that are changing the world, the way we live, the way we communicate, the way we travel and I appreciate them all. We need people like that. They are on the frontiers of the information age that programmers like me and many of you are riding on. It is here we start to see the difference between someone who is a programmer first and someone who is a problem solver first. I believe it is a matter of preference, experience, and the level and kind of exposure one has in the budding years in tech.

I had a couple of “tech friends” right from my junior college some 4 years ago, people with whom I could discuss programming and tech in general. Most of my collaborated projects were with these people, and we always worked on something because we enjoyed it, purely out of our “passion” for computers. We spoke about new languages, technological advancements and people in tech in our free time. We never thought any of this would help us get a good job or any project would appear on our CVs. That is one reason why technologies and frameworks are scattered all over my blog, instead of quality projects that people actually use in one particular technology that I could’ve been good at.

So that’s what I’ve learnt recently, and wished to share here. I (or anyone) won’t know which way is the right way to go, or if there is one, and it doesn’t really matter much as long as you enjoy what you’re doing and make a decent sum of money doing it. I admire people who are passionate about programming as well as problem solving, and the world would really be incomplete without either one. Thank you for reading.

Book Review – Deep Work By Cal Newport

Uncategorized

I watched this video on Youtube that had a list of 10 books that every student should read. The first one, Deep Work by Cal Newport, was of particular interest to me, and I decided to read it. The reason? I’m a distraction junkie.

Introduction

On the highest possible level, the book is divided into two parts. The first part is where the author tries to convince us that deep work is valuable, that it results in a greater degree of productivity, that many successful people who create value embrace a state of hyper-focused concentration towards the work they’re doing. The book illustrates how an obligation by employers (especially in knowledge work) to respond to emails quickly, be active on social media websites and to work in a common work space full of noise, interruptions and shallow tasks (like frequent meetings) has resulted in a generation of knowledge workers who have accepted distractions as a way to life.

The second part assumes that we’ve accepted that the deep life is a good life, a life well lived and gives us a set of guidelines that we can follow to get the most out of our work hours. The reason for this part is quite clear. If I was asked to switch off my email for 3 days straight, or if an employee decides that he/she will not participate in their team’s daily meetings, that will have adverse effects on our careers. Best case, people will be upset with your delayed responses to their messages, worst case, you lose your job. Now how does one strike a balance between the two parts of their work life, the immediate part (shallow work) and the important part (deep work), without sacrificing a hand and a leg in exchange.

Following are a few key takeaways for me from the book. I’ll try to summarize each point in a line or two. The quotes are from the book itself.

Need for Deep Work

The author mentions the three groups of people who will thrive in the current economy, where most of the jobs are getting replaced by computer code.

  • those who can work with intelligent machines
  • those who are the best at what they do
  • those with the capital to invest

Now unless you belong to the last category, you’ll need to produce high quality work at an elite level. A lot of people can write computer code. But the few who create something of value with code that the masses then use is elite work. Deep work isn’t for those who wish to live an ordinary life. You can be a good engineer who does what the job expects from you and never need deep work. Deep work comes in when you want to push the abilities of your concentration muscle to its limits and produce high value in your profession.

Now depending on how we look at it, due to the social media / distraction media outburst, the number of knowledge workers who can work deeply has reduced significantly. It opens up a whole new opportunity for people who still have the ability (or can train themselves) to do deep work. And deep work is still valuable. Look at all the various artists, painters, musicians and other creative workers. It is common to see deep work in their profession because of it’s immediate effects. Deep work gives these people a sense of meaning in their work. But in case of knowledge workers, Internet (and the associated distractions) is an inevitable part of their lives, and that is why deep work is becoming increasingly rare here.

I have been a happy man ever since January 1, 1990, when I no longer had an email address. I’d used email since about 1975, and it seems to me that 15 years of email is plenty for one lifetime. Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration. – Donald Knuth

Embrace Boredom

Idleness is not just a vacation, an indulgence or a vice; it is as indispensable to the brain as vitamin D is to the body, and deprived of it we suffer a mental affliction as disfiguring as rickets… it is, paradoxically, necessary to getting any work done.

My personal problem right here. When I get 10 minutes of free time, I look into my phone. I have tonnes of applications that then keep me engaged (“updated”). I have content aggregators like Reddit which make sure I’m never short of content that can quench my thirst for distractions. And this is while I actively realize that the stuff I’m reading isn’t of much value to my future. To use an analogy, it is like having street side “Chinese” food. You know it isn’t good for your health, but you end up giving in to the temptations. Now, the obvious demerit to this is an adverse effect to your health. But what about the effect it has on your will power. Giving in to temptations and desires weakens your “mental muscle” for will power. It becomes harder and harder to resist the temptation when you keep giving in over and over again. The only way to train this muscle is to actively resist the temptation for distractions.

And apart from this, giving in to easy distractions keep your mind busy and dries it off its ability to work on value generating problems. It does not let quality thoughts come into your mind for subconscious processing, and as a result, you slowly but steadily lose the power to focus and do deep work altogether.

(Quit) Social Media

The author talks about all the adverse effects social media has on our productivity. Having a “presence” on websites like Facebook and Twitter, posting regular updates to keep your “audience” engaged and thus boosting whatever business you’re in, is not only a very misinformed thought, but also something that has become quite common in the information age. Here the author makes some great counterpoints on why, even if social networking has some benefits to it for some people, most people are better off spending the same time on work that can earn them the respect and reputation that can outweigh any benefits of a social media presence in the first place. Not to mention, this can save you the thoughts about “what your friends might be doing in their life right now?” when you’re working on something important.

Miscellaneous and Conclusion

The author talks about (surprisingly) detailed ways to get back your time from shallow activities. From talking to your boss explicitly about the ratio of your shallow to deep work, to answering an email such that it does not generate follow-ups that will take more of your time, to simply filtering what emails to actually answer. These are some real life tips that can be embraced by many knowledge workers.

I think this was one of the best self help books that I’ve read (I’ve not read a lot of them, to be honest), just as useful as “How to win friends and influence people” was for building relationships. The author’s use of simple language, of logical arguments (that most of us think about, but never question or go a step further into researching), examples of strategies, real world case studies and how to practically apply them into our “busy” schedules are some of the reasons that make reading this book truly worthwhile. I would highly recommend it to all those who feel “busy” all day, but don’t get a lot done. Who feel the need to check emails, messengers, social networks every 30 minutes to not miss out on the important stuff. Who want to overcome their addiction to shallow irrelevant “news articles” and memes. I’ll leave you with this…

Ironically, jobs are actually easier to enjoy than free time, because like flow activities they have built-in goals, feedback rules, and challenges, all of which encourage one to become involved in one’s work, to concentrate and lose oneself in it. Free time, on the other hand, is unstructured, and requires much greater effort to be shaped into something that can be enjoyed.