Holy sh**t, double pointers!

So, double pointers; if you haven't read my previous post Pointers with *, &, ->, . and you don't already have a good understanding of pointers; I would really suggest going to read it. If you're a little rusty, you should probably go back and check it out anyway as it'll give you a good boost. Regardless, holy shit, double pointers what in the hell is a double pointer and why would I ever want one?

Now I could do the normal explanation and just say that it is a pointer to a pointer. But wait, what in the hell does that mean!? This is the big question that I had when I first got started in pointers. So let's see how to define a double pointer.

int  **x;

Really? That's it? It cannot be that simple; oh but it is. Now the fun comes; what happens if I dereference x? We get an int pointer. HUH!? I thought when we dereferenced a pointer we got the type back! Remember that when we dereference a pointer we get the variable back that the pointer referenced. Oh my god, here he goes again making no sense! STOP and take a deep breath. Think about it; what is a pointer? It's a variable that holds a memory address.

 _______________
| 8 bytes, s**t |
 ---------------
^ 0x01     0x09 ^

So let's assume that we have a variable x that is a pointer. That pointer contains the address 0x01; so when we try to dereference that pointer *x; we end up with the variable that exists AT 0x01. So where does x live? it IS a variable right? So that means that it MUST have a memory location!

 ______________
| 4 bytes, ugh |
 --------------
^ 0x11    0x15 ^

So let's assume that our variable x lives at 0x11. And inside of those 4 bytes is the memory location 0x01. Now here is the cool part; what if I wanted a reference TO x? Then you need a pointer to THAT pointer; and hence is born the double pointer. BRAIN EXPLOSION! You'll be fine, let's not get into the WHY would I do this; but rather the how does it work. So let's say we have a the following.

int  i = 10;
int  *x;
int  **y;

As we saw from my previous post, x = &i; means that *x and i are the same variable. This means that if we do *x = 20, then we know that i = 20. Now here is the mind blowing part, y = &x means that *y = x and **y = i. OH MY GOD!? Holy crap, how can that be? Remember each time you dereference a variable you treat the contents of the variable as a variable itself. The only reason that we know it is an integer is because we define the pointer as an int *.

Ok, so that makes sense; I can have a pointer that points to another pointer. But the big question is WHY? Why would I ever care to do this!? Well there are two main reasons; the first and biggest reason is multidimensional arrays, another is to create iterators, and also to create variables. Remember in the previous post we said what? Pointers are used for dynamic arrays. So having a dynamic array of pointers is a dynamic array of dynamic arrays. Let's look at this.

        V 0x01  V 0x09
         _______
LAYER 1 | 0 | 1 |
         -------
        V-^   ^-----V
         _______     _______
LAYER 2 | 0 | 1 |   | 0 | 1 |
         -------     -------
        ^ 0x09      ^ 0x17

So I denoted these as layers, essentially you are dealing with a variable such as var[LAYER1][LAYER2]. This means that you have an array of 2 members; each of those members containing an array of 2 members.

So what does the memory layout look like? Well for the variable var, we are actually a pointer that contains the memory address 0x01. Here we can reference either member 1 var[0] or member 2 var[1]. Each of these are pointers that contain memory addresses. For example, if we reference var[0] we have a reference to member 1 var[0][0] or member 2 var[0][1]. OH SHIT!! This is freakin' awesome; being able to create dynamic multi-dimensional arrays is freakin' fantastic!

But wait... There's more! So what about iterators. The biggest way to show this is through an iteration over a character array. Remember that char *'s are C versions of strings. So now, what happens if we wanted to send a character pointer and move to the next space. So we take a pointer to the head of the char *, then we pass the memory location of the new variable to our function.

char  *c = "Oh damn, this sucks!";
char  *cp = c;

Remember that cp will just hold a pointer to the variable c. So let's look at two things, first is the calling of the function.

next_space(&cp);

As well as the function definition.

void  next_space(char **p) {
  for( ; **p != ' ' && **p != '\0'; (*p)++ ) { }
}

HOLY SHIT! Wait a second though, why does (*p)++ work? Easy, we dereference from the double pointer p, and we increment the original variable cp from the caller. How does **p become a character? Oh right; because * dereferences a variable, then we end up with the original cp[0] variable since it is the same as **p!

So you had said creating variables; how does that work? Oh yes, this is pretty cool, terrible stuff! I say cool because you can create API's that utilize them quite well; terrible because if you don't do it right or if people don't understand what you are trying to do they can create HUGE memory leaks. Let's look at an example. We have a pointer char * because we want to store a string.

char  *c;

Now let's assume that have a general string that we want to store into c, we have a function store_string() that we call like below.

store_string(&c, "Hello World");

So now let's look at the store_string definition.

void  store_string(char  **ptr, char  *str) {
  *ptr = malloc(strlen(str) * sizeof(char));
  int i = 0;
  for(i = 0; i <= strlen(str); i++) {
    (*ptr)[i] = str[i];
  }
}

OH SHIT!? Yea that's right! Now here is the best part; we can clean up the variable and set it to NULL FOR THE CALLER!!

clear_string(&c);

So what could this awesomeness look like!?

void  clear_string(char  **ptr) {
  free(*ptr);
  *ptr = NULL;
}

OH MAN THAT IS THE GREATEST SH**T EVER! Yea, yea; here is the key to double pointers and of course ANY pointers is knowing when and where to use them. I've seen double pointers used in some of the worst places ever; as well as pointers. Remember the less you deal with allocating the memory for the pointers; the less likely you are to create large memory leaks! And don't expect me to feel sorry for you if you try to create a "cool" API that takes in a double pointer and generates your multi-dimensional array and you end up loosing tons of memory. Remember, YOU NEED TO FREE EACH OF YOUR POINTERS THAT YOUR POINTERS REFER TOO!

My point of saying this isn't to scare you; but to ensure that you are thinking up a good use case for creating the double pointer!

DISCLAIMER
I am not going to cover single/double void pointers; if you want to use void pointers you need to learn how to do casting and you really need to figure out a way to determine what datatype you are dealing with in the void pointer. Why? Because you cannot dereference a void pointer! This means if you have void *ptr; with an array of int's you cannot say ptr[x]. Why? Because you cannot dereference the void *, because void has no size so there is no size for the offset of ptr. If you said ((int *)ptr)[x] then it will work because you've changed the data type from (void *) to (int *). We will cover pseudo-inheritance in C probably in the next post.

I find that I am doing better posting when I am drunk; I find that I am much more outgoing and much more social when I am drunk. As such, I'll be blogging while I'm (at least partially) drunk so that I can make the information fun and exciting! If you find that I am boring while I'm drunk; please let me know and I'll be sure to try harder on the next blog!

Pointers with *, &, ->, .

This is one of those things that never gets old; discussing pointers. With my heavy C background and my insatiable desire to program close to the hardware; I would say that I love pointers. But most people, when confronted with pointers, tend to loose interest and also their minds trying to understand what they mean. So here, I'm going to try my best at explaining pointers in a way that makes sense; for this I'll be explaining pointers from the reference of C.

First let's cover some background on variables, definition is pretty straight forward. Below you can see how to create an int variable.

int  x;

So let's get into the thick of it, what is a pointer? A pointer is a variable to a memory location! Wait, that's it? Is it possible that I am misconstruing what a pointer is? Not at all, instead the confusion comes when you begin explaining how to USE them. So let's see how to define a int pointer.

int  *x;

Yes, that is it, I've defined a pointer; the variable is interpreted as such; x is a variable that contains a memory location to an int. But wait, that cannot be it; people tell me that pointers are really difficult to use. Yes, it is a little bit harder, see you have to dereference the variable x to get the integer that is pointed to by the memory location stored in x. Ok, so now your eyes have glazed over right? You've decided to continue on to some other blog and see if someone else can explain it better but STOP and take a deep breath.

Let's look at a better example from our pointer variable x. Let's assume that we have a section of memory that is the memory representation of an int (4 bytes long).

 ---------------
| 4 bytes, woot |
 ---------------

Let's also put some byte numbers to this.

 ---------------
| 4 bytes, woot |
 ---------------
^ 0x1       0x5 ^

Ok, ok so what about our int *x? What the hell does that even mean and what is contained in there? Well, are you ready? It contains the address of our int (the 4 bytes above). So wait, x = 0x1? That's right! But now you have to "dereference" x. So bare with me; you have x which contains the memory location of our int. We need to get THE int. Therefore we need to get the data AT the memory location right? HA, that is dereferencing. So how do we do it?

*x=10;

No, it cannot be that simple... Yes, actually it is! The * tells the compiler to treat the variable as a pointer and to use the current variable as a memory location. So how do we know that it is an int? Aha! Because we defined it as an "int *" which means that inherently when we dereference x, *x must be an int. So now that we understand that the * indicates a reference and is used to dereference a pointer; what the hell is that ampersand '&' used for?

Well, the ampersand is used to indicate "reference of." Wait, I thought that was what * was used for. No, I said * was to indicate a variable is a reference or is used to dereference a reference. The & is used to say the memory location (reference) of a variable. Let's see this from our original variable x (I shall refresh your memory).

int  x;

See how x is just a nice happy variable? Not a pointer! So let's look at some more memory locations. It is important to note that since x is not a pointer, the memory layout below IS the x.

 -------------------
| 4 bytes, oh noes! |
 -------------------
^ x IS here

So when you specify &x, what are you saying? You're getting the memory location OF x! WHAT!?!?! That doesn't make any sense! how can you GET the memory location; yea I know you want to switch blogs again STOP and let's take a deep breath.

Remember that x IS a variable and therefore the variable MUST lie in a memory location. So when you do &x you are merely getting the memory location of that variable.

 -------------------
| 4 bytes, oh noes! |
 -------------------
^ 0x1           0x5 ^

So why would I want to do this? Because in order to create a pointer, it must reference a valid memory location. Check this out.

int  x;
int  *y;
y = &x;

OH MY GOD, MY HEAD EXPLODED! Nah, it's not that bad; until of course I tell you that *y and x are the same variable. If you don't understand that, then check it out, let's say that x is at location 0x1 (taking from our previous example). Then when we get the variable location 0x1 and store it into a variable that HOLDS the memory location (y), then when we dereference (there is that stupid ass word again!) *y we end up with an int at the location 0x1. OH SHIT, THAT IS THE LOCATION OF x!!!! GET SOME!

So what happens to x if I do *y = 22? Right, x = 22! Why is that? Because *y and x are THE SAME VARIABLE! I'll say it again if you want me to but I'm sure you're getting tire of me explaining it; so I'll say this if basic pointers don't make sense, you should start reading this over again (if you don't understand iteration, you will by the end of this).

Fantastic, so now we understand the basics of pointers. But what about those stupid ->'s and when can I use .'s? Ok, we're getting ahead of ourselves. So if you know about structs (if you're doing anything in C I'm making the assumption you've heard of them) but if not let's look at an example.

typedef struct str{
  int x;
}str;

So here we have a basic struct; essentially it's like a class with only public members (and no methods) in Java. If you don't know what a class or a struct is; you shouldn't be messing around with pointers. So let's look at referencing the variable x inside of an example str.

str s;
s.x = 10;

Well that wasn't too bad; you use the . operator to indicate a member of the str struct. But here is the thing; let's say that you wanted s to be a pointer. Why? Well, we'll get to that; but right now just follow (I haven't steered you wrong yet!). Let's assume that our variable s is declared as below.

str *s;

Then we want to access x. Ok, simple right, s.x; no... s->x; oh holy crap you're confusing me! Nah, it's really not that bad, s was a pointer right? So what made you think that you could just access x as if s was the variable!? Oh I get it, s is a pointer, which means we had to dereference it to get x. So that means that -> dereferences s and gets the variable x from the struct! Right! if you wanted to write it (and seem cool) you could do (*s).x but no one would hire you because that would be horrible to read everyday.

So wait, that's all the -> operator means? Yes! See, I told you pointers weren't that bad. But of course, there is the big question; how do I actually USE pointers? Dynamic array sizes. What does that have to do with anything? Easy, the function malloc() which allows you allocate a section of memory. Let's say I want to get an array of 2 integers.

int * x;
x = malloc(sizeof(int) * 2);

If you don't know malloc, you'll need to go check out the documentation but essentially we are just allocating the size of two integers (8 bytes) in memory and getting the memory location. Storing it into a pointer. What does this look like in memory?

 ------------------
| 8 bytes, badness |
 ------------------
^ 0x1          0x9 ^

And of course x holds 0x1. Now, since we allocated 2 ints it is important to note that THERE ARE TWO OF THEM. So how do we get them? i[0] references the first i[1] indicates the second. WAIT, huh? Ok, so i[0] dereferences i and then goes to an offset of 0. i[1] dereferences i and then goes to an offset of 1. So where does this offset come from? Easy, the offset is defined by the SIZE OF THE VARIABLE TYPE! Since our variable type is int, the size is 4 bytes; so i[1] will go to 0x1 + 4 = 0x5.

So can I name another reason to use them? Sure! If you wanted to modify a variable without having to return it. What? That makes no sense. Sure it does, let's say we want to take in a str variable (again stealing the definition from above).

void modify_str(str * s) {
  s->x = 10;
}

OH SHIT! That makes sense, yes! Is that awesome or what!? So here is the cool part; returning pointers. Example, let's say that we want to create a new str, and set the internal x to 1983 (my birth year because I'm picking random numbers).

str *new_str(int  i) {
  str *s = malloc(sizeof(str));
  s->x = i;
  return s;
}

OH DAMN! This makes initializations work! All of this is starting to make sense; why we would need to use pointers, how pointers work, and also how to effectively use pointers. Yes, and it really isn't as bad as many people tend to think. So here is the thing, pointers are pretty straight forward; double pointers and higher become much more abstract.

DISCLAIMER
I am not going to cover double pointers; but since you understand pointers now; just think of them as pointers to pointers (since that is what they actually are). And more importantly, you really won't use double pointers until you start needing to do pointer arithmetic in order to do iterations.

I'll say that I'll cover double pointers at a later time; but I have no guarantees that it will happen. If it does, hopefully I will be as fun and exciting as I have been so far! Just remember, pointers store memory locations of variables.

How can you not use <insert favorite IDE here>?!

This is one of those subjects that really bugs me is when people promote an IDE to the point that they are insisting that you are a complete failure for not using that IDE. So I realize that people always have an IDE that they really like. Mine is Eclipse for Java development; however, for C based applications I prefer to write in a text editor (vim is my poison). I have my own reasons, which I will outline below, for why I like these two setups.

Eclipse I like for Java style development, it has some really great auto-complete. However, I also like the simplicity for adding external jars and linking projects together as dependencies. Also there are quite a few good plugins for things like Ivy and Subversion that give tons of extra functionality. Why not Netbeans? I've never been impressed with the auto-complete and the auto-populate for import statements. Also I hate the fact that most class names will be fully qualified which bugs me (this may have been changed in a more recent version but this is how it was in a previous version).

Vim I like for C based applications because I've never found that any IDE that gets out of the way and has a good built in debugger. Frankly when I'm writing in C I really don't need that much help since with Vim I get CTags. This gives me the ability to investigate a function on the fly as well as investigating structs.

I realize that I am putting forth these statements and it sounds like I am breaking the rule that I put forward. Well not quite, I'm not stating that anyone else should work the same way that I work. Instead I am stating the reason(s) why I am doing something. The thing that I cannot stand is when I say "I use Eclipse for Java" and someone else goes "you should really use NetBeans, it's the awesomest and you are an idiot for not using it." I've also heard the argument "well you don't use an IDE anyway (for C development) so why not use something?" Why not? Because I'm more effective in vim, I have very few problems within my type safety so I don't need an IDE checking my type safety everytime I finish a word.

So what exactly is my point? Well, I'm not saying that you shouldn't propose someone using an IDE that you are used to; instead I'm stating that you can propose an IDE, just don't push that idea to death. It annoys me when I am working in a specific environment and doing well with it; and someone comes along and goes "well boys, we're going to change things up and go with this IDE, everyone should now be using it." Get off my back, I do more work in 2 days than you do in a week; maybe you should adopt my environment because it seems to work. Oh wait, you have problems working in my environment? Then why would you push me into an environment that I might have problems working in?

How much is too much dereferencing?

This is mainly focused in any language where you can have composite classes or structures. Just to explain this, where you have classes inside of classes or structs inside of struct. Consider the following.

C Example with single inside struct

struct inside {
  int i;
};
struct outside {
  struct inside i;
};
int main(int argc, char *argv[]) {
  struct outside o;
  o.i.i = 10;
  return 0;
}

Does this really make sense when we look at

o.i.i

? If it does, let's look if we changed inside like the following.

C Example with deeper inside struct

struct deeperinside {
  int i;
};
struct inside {
  struct deeperinside i;
};

Now, we call the internal i as o.i.i.i which becomes pretty unruly. Obviously using the same variable name each time would be pretty terrible. But even changing the name doesn't always help. Let's see what happens if each element is actually a pointer.

C Example with deeper inside struct pointers

struct deeperinside {
  int i;
};
struct inside {
  struct deeperinside *i;
};
struct outside {
  struct inside *i;
};
...
struct outside *o;
...

So now we reference the int i from deeperinside as o->i->i->i This is completely unreadable, so let's see what happens if we at least add some casting.

C Example Access to deepinside variable

((struct deeeperinside *((struct inside *)(o->i))->i))->i

Although at least you can understand this, it becomes completely unreadable; so what is the answer here? How about we create some temp variables that can be used to store our inner variables?

C Example Access to deepinside variable through temp variables

struct inside *tempinside = o->i;
struct deeperinside *tempdeeperinside = tempinside->i;
tempdeeperinside->i

Notice that this becomes much more readable and understandable. This allows the people who will be maintaining the code much happier and faster to find/fix issues as they may arise. Now I've been focusing mainly in C but this also applies to languages such as Java. Why Java? Because A) it's the language that many programmers are programming in today; and B) it's much easier to encapsulate classes inside of classes (and is usually encouraged). By bringing up the idea of how we deal with accessors to these inside classes we can ensure that people at least understand how they can better display them to make them more manageable.

Another option in an actual OO language would be to encapsulate those objects, this in and of itself means that those o.i.i.i would turn into o.getI().getI().getI() which again makes no sense; are those composite classes (classes that encapsulate a class of the same type)?

Remember, making code reusable is good; making code understandable is much more important!

Do you like programming or YOUR programming?

This is a very interesting question that I've seen come up again and again; although I've never know the best way to express it until now. Out of all the languages listed below which ones can you not stand? If presented with that specific language to keep up; would you do it or would you quit? The question is, do you like Programming or do you like YOUR programming?

Perl

sub Example($) {
  print $_[0] . "\n";
}

Lisp

(defun Example (var)
  (print var))

C/C++

void Example(char *var) {
  printf("%s\n", var);
}

Java

class Temp {
  public void Example(String s) {
    System.out.println(s + "\n");
  }
}

If you looked at any of these languages and you said to yourself "I'd never program in that language" then you've just answered the question. Many of us developers are presented with a code base and are asked to make changes or to fix it. We normally don't have the ability to tell the client "well I can change it to language X in about 5 weeks if you'd like" because the client will say "what do I gain from this?" The answer is; well you get nothing extra but it will be easier to maintain. This then presents you with a big no because the client isn't going to pay you to rewrite an already running application into a language that you want (mainly because someone else will come along with this same statement).

We all have languages that we like; we all have languages that we're good at. This is true, but being scared of a specific language to the point that you would leave a job instead of program in it is just ludicrous. My point is not that you should seek out work in languages that you are not familiar with; but instead be open to working in different languages. The first step to working in an unfamiliar language is to find out how to do things that you would normally do in your known languages. NOTE: I do not condone going out and buying a book on the language and trying to cram it all in! Instead pick up on new ways of doing things as you are programming.

One of the big pitfalls that I've seen is someone coming into a team with programming knowledge. They then try to apply their previous programming experiences to a language that you might not know. Because this is the case, always listen to your teammates on how to better work in the language. Also, if you believe that your way of programming is getting ridiculed and you feel it is better than the way being suggested you should stand up for your way. Now be open to change and listen to why someone is making the argument that your way is bad/inefficient etc... Make sure to get them to explain why their way is better (and yes understandability is a feasible reason for example a .foreach() method rather than for(int...) {}).

So why bring this up? I have a coworker who is a DBA in MySQL who is going to be leaving. They have always been a huge proponent of DB2; always trying to get the entire organization to DB2. Now, the company I work for is not a small 50 person group; I work for a medium 500 person who just got bought out by a 100k person corporation. The problem with pushing these types of changes is that once you grow so large the company is not going to do a massive huge change just because someone feels that they can better manage a DB2 instance. This would require every application we use to be changed from MySQL to DB2 connectors which would be quite a large problem since almost every application we have developed is in C. We already have MySQL libraries that are well tested and work well; so having to switch to DB2 would be a huge change that the company was not wanting to risk.

So now my coworker is leaving because they didn't get their way. So how does this relate to my original example? Well think about this; it's not just dealing in programming where you shouldn't be afraid of working with new languages. Instead, you should always be open to work in new applications and expand your horizons. You come from a completely Windows shop and get hired as a Linux administrator. Don't be afraid, pick up that Linux administration book and install a copy at home. Play with it, break it, fix it, learn it. The way I see it, we as technology professionals should always be willing and eager to learn about new features in the technology field and not just limit our learning to ONLY our specific field. That's right, if your a Linux admin go learn something about networking or Windows administration. If you're a developer, go learn something about networking or host administration. If you're a DBA who works with DB2, go pick up a copy of Oracle or MySQL and see how they compare; maybe you'll find something interesting from that other piece of software that you would like better.

Unit testing the proper way

Many times I've seen Unit tests that really don't help out. You will often find Unit tests that will quite often be like

public void TestNullInput() {

  MyObject obj = new MyObject();

  obj.functionThatShouldNotFail(null);

}

This really doesn't help that much; now it does need to be done especially in languages such as C or C++. You must always check what happens if you pass a function a NULL pointer. However, many times we see these that people will put in Unit tests for the most obvious tests. But what happens what you end up with the more non-obvious tests? Look at this example below and try to figure out what we're testing for.

public void TestInput() {

  MyString str = new MyString();

  str.parse("Hello|world!"); 

  Assert.assertEquals(str.parsedCharCount(), 

    "Hello|world!".length());

}

Well, it's really not that obvious at first; we just think oh why would we care about checking for the parsed amount of letters. Well here's the thing; let's assume that we're doing parsing and we need to do something special when we hit the '|' character. Let's assume that we are skipping it or something to that effect; why would we be looking for 12? We if we think about C/C++ (not so much managed languages but we'll assume that it might happen in their case as well) then it is very easy to run off the end of the array. So what we do is to count the number of characters that we are parsing; this allows us to check and ensure that we ONLY parsed the number of characters that was possible in the string (the max length of the input string).

The idea isn't just to throw out tests like this in every possible case before declaring that your code is ready. The point of bringing this up is that when you come across an error or a segmentation fault etc... there is always a way to write a Unit test to check for an error. Unit tests provide a way to detect bugs that may have regressed; and because we can detect them when they regress it can provide us a way to fix problems that we may have resent out to the consumer.

Developing scared software

This is something that I've begin to notice and have a real problem with. Now, it may not be you but you know SOMEONE that has done this. A problem/issue arises or there is a deprecated library call that is being used. It is something that has been working for quite sometime but might be either holding back new features because it is no longer supported; or it's not broken enough to force someone to fix it. This is where the people come in and say "well this needs to be fixed" but they never make any changes to it.

Sometimes this is accompanied by a "it's in a library" or is followed up with "we'll fix it at some point in the future when we have to." So why is this an issue with me? Easy, it's a lame excuse; if you see that there is a problem YOU, as the developer, should be man enough to step up and fix the issue that is starting at you. Yes, I do understand that there are processes and bugs that need to be prioritized; but if you see that it is an issue then you should make the case to find time to fix it.

I've seen this from a few different developers and it kills me when I hear it. "Oh yea, I'd love to replace that." Really? If you would love to do it, then do it. Especially if you see the problem and you know what the acceptable fix is; this means that you've already thought out a plan to fix the issue and as such you should put your fingers to work fixing the issue. Your users may not thank you, but your peers will once they see that you're working to make the product easier (hopefully with your fix) to maintain.

The reason that I call this scared software is that the software is coded, almost on eggshells, in a way that no one really wants to make big changes to it. Thus, to get a new feature (or to perform a bug fix) many people will end up writing tons more code to work around the issues that no one wants to fix. The solution is simple; if you see an issue or see an improvement, do it.