If pointers are so "simple", why are they so confusing?!? Because pointers are one of those concepts that's easier to use than to explain.
Some concepts like pointers (and also monads and also English words like "a" and "the") are easier to teach by examples instead of by definitions.
E.g. Consider the English words "a" and "the": a 5-year old can fluidly use those connecting words when talking but a 50-year old adult that's been speaking English for decades will have a tough time defining what the words "a" and "the" _mean_ without giving a confusing and circular explanation. Attempted explanation: the "a" is the indefinite or general signifier while "the" is the definite or specific signifier -- what's the problem?[1]
How did small children learn how "a" and "the" work without being given a formal definition? Because of numerous examples.
By working through enough examples, the lightbulb moment happens ("oh gee, is that all pointers is?!?") and the learner is able to derive his own definition. But the irony is that the learner now has The Curse of Knowledge[2] and he/she now thinks he can write a "better" tutorial to explain pointers but new learners reading that tutorial will still say "I don't understand your explanation at all."
> pointers are one of those concepts that's easier to use than to explain.
My experience is the exact opposite.
A pointer is just an address for something. There is a "reference" operator to get a variable's address (pointer) and a "dereference" operator to get the data at a given address. That's it.
The hard part for me has always been using them correctly - keeping track of what points to where in my head, especially for void pointers. I haven't done any significant development in C since my early programming years, but that's the part I remember struggling with.
They're sort of like GOTO in that they're easy for me to understand, but they can quickly make my code hard to follow.
>A pointer is just an address for something. There is a "reference" operator to get a variable's address (pointer) and a "dereference" operator to get the data at a given address. That's it.
Everybody's brain is different but your exact "simple" explanation for pointers has been tried on many beginners and they still don't "get it". It's still going to be confusing.
For many (most?) beginners, it's only after they've done some examples when they can go back and re-read your simple explanation such that it makes sense. Therefore, your simple "definition" becomes understandable after the person has learned what pointers are through examples.
But then you need to explain to a newcomer that you can have `int ptr;` (pointers to pointers)... It's pretty odd for someone coming from say, Python, Java, or JavaScript. (which, many go through years before realizing that pass by reference happens in their language too)
> you need to explain to a newcomer that you can have `int ptr;` (pointers to pointers)...
I think HN swallowed the asterisks there. I presume you meant:
int **ptr;
> It's pretty odd for someone coming from say, Python, Java, or JavaScript. (which, many go through years before realizing that pass by reference happens in their language too)
> For objects, the pass by value is the value of the reference to the object.
That's exactly what "pass by reference" means. According the SO definition, C is always pass by value: the value of the 'primitive' (int, long, char) or the value of the pointer (an address.) It pretty much applies to any language: ultimately you're passing a value from the caller to the callee. Whether that value is 'raw' (say, an int) or a pointer or reference is all in how you (or your compiler, or your runtime) interpret that value you received.
No, that's wrong. Pass-by-value with references as values, is not the same as pass-by-reference.
Plenty of people make the mistake of equating the two. They can get away with the misconception as long as they don't encounter a language like C# that features both reference types and optional pass-by-reference.
> According the SO definition, C is always pass by value
That's correct. C is always pass-by-value. You can get a similar effect to pass-by-reference, by passing a pointer.
> It pretty much applies to any language: ultimately you're passing a value from the caller to the callee.
That's incorrect. If the language has pass-by-reference semantics, then it does not copy the value, it passes the callee a reference to the variable. It's not true of pass-by-name, either, but don't know much about that strategy.
A good article on this topic is the documentation on C#'s ref keyword, which enables pass-by-reference. The article does a good job explaining the related concepts. [0]
The acid test for whether pass-by-reference is happening, rather than pass-by-value, is whether the asserts can fail in code along the lines of this Java code:
int myInt = 42;
doStuff(myInt);
assert(42 == myInt); // Ok
String myString = null;
doMoreStuff(myString);
assert(null == myString); // Ok
In Java, the asserts will never fail. The two methods called do not have access to either the myInt local or the myString local, they only have access to copies of the values held in the two locals. This is the reason Java would permit us to use the final modifier when declaring the two locals, without needing to know anything about either of the called methods.
On the other hand, C# permits pass-by-reference. Here we'll suppose that both doStuff and doMoreStuff declared their parameters using the ref keyword:
int myInt = 42;
doStuff(ref myInt); // If callee uses 'ref' keyword, caller must use it too
assert(42 == myInt); // May fail
string myString = null;
doMoreStuff(ref myString);
assert(null == myString); // May fail
In this case, either assert may fail, depending on the behaviour of doStuff and doMoreStuff. The callees are given references to our locals, and so they may mutate our locals (as well as reading their values of course).
Related to this, this is illegal in C#:
doStuff(ref 42); // Compile error. 42 is not a variable!
I remember learning Java when I was beginning programming and told that it was "pass by reference except for primitive types which are copied", when that's a total lie! It's not pass by reference at all. I think they said this because C++ copies objects (or nowadays, moves them) when you pass them as arguments (unless you pass by ref), and they wanted to make the distinction with C++ (i.e. "Java doesn't copy objects when you pass them to functions"). But it's done a terrible disservice to the entire industry confusing programmers for years by using the term "pass-by-reference", which has an entirely different meaning.
What people mean when they say "Java is pass-by-reference" is that "if you pass a mutable object to a function, the function can mutate it". But that's not the same thing at all.
I encountered similar confusion in faculty members as an undergraduate.
To resubmit what I wrote in a comment below:
It's confusing that we use 'refer to' in two different senses. In Java, a reference-type variable 'refers to' an instance, but overwriting the variable does not impact the referred-to entity. In pass-by-reference, a parameter 'refers to' a variable, and overwriting does impact the referred-to entity. Perhaps we should instead say that in Java, a reference-type variable 'points to' an instance.
> ...it passes the callee a reference to the variable.
And that reference itself is a value-- it's a thing passed to the function and copied from somewhere in memory to a register. It's the interpretation of this value that makes it a reference to some other tangible item. Further, it's the implementation details behind the syntactic sugar, compiler optimizations, and runtime support that allow the developer to reason abstractly with references.
Your Java example with String overlooks the fact that String is immutable: the object itself can't be modified. Use any other object type, allow doMotreStuff() to operate on the argument's properties, and you get the same effect as if passing by reference.
My point is that the SO explanation isn't a good one precisely because you get this pass-by-reference behavior in Java. Yes, indeed, technically there's this value that gets passed to a function, and it's the value of the reference. And, yes, everything that "passes by reference" has to pass something and that something is the "value of the reference" so everything is technically, at the bottom of the abstract stack, "pass by value." But that's not helpful when attempting to help the newb understand why his object mutated in a method when "pass by value" is what he expected.
"call-by-reference" already has a technical meaning, and it is widely-misunderstood because "reference" is an overloaded term.
Variables are cells that contain values. A better name for "call-by-reference" semantics is "call-by-variable", because the variable -- the value-containing cell -- is what is semantically received by the called function. Moreover, it doesn't matter how this is actually implemented, so long as a caller may effectively rebind a caller's variable.
In Java, variables may contain either "reference values" or "primitive values". It is never possible, in Java, to rebind a caller's variable. (Consider that there is never a call site that could be rendered illegal by making a non-final argument final.) We should not call it "pass-by-reference", not only because it's incorrect but because, to your point, it's entirely misleading when "reference" already has a distinct meaning in Java.
Pointers are strictly more powerful in a language than pass-by-reference. If you have pointers and pass-by-value, you can emulate pass-by-reference by providing a pointer value that points to a variable cell. So if you have pointers already, you don't strictly need call-by-reference.
C# doesn't have pointers in the managed subset, because raw pointers can't be managed. But call-by-reference can be added relatively easily, so you can recover a small amount of what could be done if you had pointers. That's the purpose of the `ref` keyword.
Java doesn't have pointers, either, but it also doesn't have call-by-reference. That makes it impossible to write the canonical example of call-by-reference, `int a = 1; int b = 2; swap(a, b)`. Instead, you must reify the concept of a cell in the first place, e.g. by wrapping each value in a one-element array, and pass those cells to `swap` instead.
> that reference itself is a value-- it's a thing passed to the function and copied from somewhere in memory to a register.
I don't see your point here. Java always passes by value. This isn't an open question. Computer architecture, assembly code, and compiler specifics, are irrelevant.
> it's the implementation details behind the syntactic sugar, compiler optimizations, and runtime support that allow the developer to reason abstractly with references.
It isn't. References types, and parameter-passing, work however the language defines them to work. The language's implementation isn't relevant.
> It's the interpretation of this value that makes it a reference to some other tangible item.
In pass-by-reference, the parameter refers to the passed-in variable, and you aren't given a choice about it. If you assign to a parameter which was passed by reference, the change will be visible to the caller. This is different from passing a pointer by value (as in C) and from passing an object-reference by value (as in Java).
(It's confusing that we use 'refer to' in two different senses. In Java, a reference-type variable 'refers to' an instance, but overwriting the variable does not impact the referred-to entity. In pass-by-reference, a parameter 'refers to' a variable, and overwriting does impact the referred-to entity. Perhaps we should instead say that in Java, a reference-type variable 'points to' an instance.)
> Your Java example with String overlooks the fact that String is immutable
It does not. The immutability of the String instance is irrelevant. Swap it out for another class type, or for an array type, and everything I wrote still applies.
> Use any other object type, allow doMotreStuff() to operate on the argument's properties, and you get the same effect as if passing by reference.
In Java, everything is passed by value, including Java's reference types. Sometimes, pass-by-value and pass-by-reference would behave the same. We could demonstrate this with a C# example. So what?
I'm not talking about accessing the members of the referenced object, as that isn't instructive here.
Using pass-by-reference would permit the reference-type variable itself to be modified. Again, it is irrelevant whether the object instance is immutable. An example where we don't use an immutable object, but it still matters that Java uses pass-by-value:
private static void doStuff(int[] arr) {
arr = null; // Only overwrites local copy. Reference types are passed by
} // value. The assignment is invisible to the 'main' method.
public static void main(String[] args) {
int[] myArr_1 = {1,2,3};
int[] myArr_2 = myArr_1;
// The value of local variable 'myArr_2' cannot be changed by this line:
doStuff(myArr_2);
assert(myArr_1 == myArr_2); // Ok.
}
> the SO explanation isn't a good one precisely because you get this pass-by-reference behavior in Java
There is no pass-by-reference in Java. It is incorrect to say that Java sometimes passes by reference. Java has reference types, has no pointer types, and always passes by value. These are distinct concepts, with accepted terminology.
> that's not helpful when attempting to help the newb understand why his object mutated in a method when "pass by value" is what he expected.
Maybe so, then we should start by explaining Java, and only explain the terms pass-by-value and pass-by-reference later on.
What we shouldn't do is teach an incorrect definition of pass-by-reference. Saying that Java uses pass-by-reference is simply wrong, and should cost a student marks on their exam.
Is that really any more complicated than arrays of arrays or structs within structs? Pointers to pointers definitely makes things harder to keep track of, but I don't think they're conceptually difficult.
> If pointers are so "simple", why are they so confusing?!? Because pointers are one of those concepts that's easier to use than to explain.
I disagree (the rest of your comment I fully agree with, FYI). What I personally found difficult is to understand that: when you dereference an invalid memory address, then the dereferencing operation is working, but the OS/runtime environment won't allow you to do it!
Wait, what? Why?! Who cares my value is wonky, why is my program giving an error? Just obtain the wonky value! (is what I thought when I first did it)
The reason it's so difficult because at this stage you're still using printf for debugging, while you really should use a debugger, to really understand what's going on under the hood. Preferably, one also learns a modicum of x86 to really see it immediately.
The concept of dereferencing the wrong values isn't explained a lot. How to troubleshoot segfaults isn't explained at all in beginner tutorials, not even in my own (I just wrote an explanation for fun as a comment in the same thread [1]).
Also FWIW, when I heard the concept of pointers I found them intuitive immediately. I tend to think and reflect a lot, and the explanation of referencing and dereferencing almost sound like some proverbial life wisdom to me.
I found their use a lot harder, in part because I found code syntax hard. In another part, because I used to find "doing things" and "building things" (of any kind) an insurmountable task.
---
With that said, I do agree with the rest of your explanation. I suppose it also depends on who teaches pointers to you. I got a classroom explanation first, a few examples and then some exercises. Then suddenly, I had to write a 500 line C program. Then all the complexity storms at you like a rabid zombie army.
But if you're used to writing small C programs all the time and picked up on pointers that way, but then suddenly have to explain it. Then, I can see how that's hard.
I think the real difference comes with the knowledge the person is carrying when learning C. For example, people that are familiar with concepts like classes, string objects, arrays, etc. brought from higher level languages are the ones that I see have much more problems when understanding memory management (and thus pointers) in C.
For example, for a higher-level developer, a string might mean some kind of object that knows its length, auto-expands, etc. When in C a string is just an array that it will be overflown at the very first chance. Same with other objects.
I find that a person that doesn't know much about languages are the ones that better understand the basic concepts of C (like pointers).
> For example, for a higher-level developer, a string might mean some kind of object that knows its length, auto-expands, etc. When in C a string is just an array that it will be overflown at the very first chance. Same with other objects.
Funny, my programming knowledge in Java helped me because of the whole concept of what a reference is and a literal value. In my first Java programming class, 8 years ago, I learned that you need to have references, otherwise things will take up too much memory. My teacher took a humongous object as an example and asked "what if I need to access this at 10000 different places, but I really only need one physical copy of it?"
I was pretty upset that you had to do all of this bookkeeping and accounting. I remember being pretty vocal about that it just didn't make sense to my why one couldn't just copy stuff. I guess we all grow up at some point :)
When I saw pointers for the first time (to be fair, years later), I was thinking "ah so a pointer is basically a reference". Other people would say "a pointer is a whole lot more than just a reference, you can do pointer arithmetic with (and so on)". And I was just thinking to myself: alright, it's a reference on steroids, but the groundbreaking essential idea of the pointer is that it refers to something, just like a reference in Java.
And then I'd hit a segmentation fault and doubt my own understanding.
Learning programming felt emotionally turbulent to say the least, haha.
My experience was nearly the opposite. I tried to teach myself c 15 years ago and found pointers completely bewildering. I didn’t really know how to program any other language then. After a 10 year, non-technical interlude, I went to grad school and spent 5 years writing matlab. In the last year, I decided to learn c again. This time, pointers seemed completely straightforward. The most difficult part for me this round was keeping the syntax straight.
> Wait, what? Why?! Who cares my value is wonky, why is my program giving an error? Just obtain the wonky value! (is what I thought when I first did it)
Program on an MCU sometime, no memory protection. Feel free to tap dance all around RAM. It is rather liberating!
Really though, pointers are IMHO easy to understand, they are the address of an actual physical location in the real world. If you have a good enough microscope, or a large enough RAM chip, it would be possible to find the literal physical location that a pointer is referencing.
Pointers are the ultimate un abstraction. They represent something real!
This here exactly. Dereferencing null. Oh maybe the compiler can figure out a pointer is null and try and ratfuck you in the name of some obscure 'optimization'. But when it can't what happens depends on the machine.
Big iron machines with mmu's will invariably throw a seg fault. Some MCU's without an MMU will either seg fault because no physical memory exists[1] or return garbage. Some machines you get interesting things like the initial stack pointer. Or a memory mapped register.
[1] Trapping seg faults and logging address of the offender takes most of the pain away.
> pointers are one of those concepts that's easier to use than to explain.
Strongly disagree. The rule of pointers is simple: an integer pointing to a memory position. However, using it correctly and effectively is hard. Dealing with pointers is like working with math. The rules are simple but there can be difficult problems coming out of these simple rules.
This article and most of the comments here are just over complicating a straightforward idea. I simply don't get why people always think "pointers" are difficult to understand; it is manipulating them to do advanced stuff that gets tricky. But nevertheless it gets easier with time and experience since you start picking up on the common idioms.
For anybody beginning C, my advice is to not make an "imaginary dragon" out of "pointers" but just study examples and write your own programs using them and soon they will seem "obvious".
I agree. I really can't understand what is complicated about them. I find the same with programmers who find the DAG in git difficult to grasp. It's just linking stuff together. It's how the the web works. It's how linked lists work. What's difficult about it.
I think exposure to assembly demystifies all of this stuff about C. Most of C is just making stuff really convenient that you would do anyway in assembly. A pointer is just a memory address, but in C it also has a type. The only thing the type is for is so the compiler knows how many bytes to add/subtract when you increment/decrement the pointer. That's it. In assembly you'd just have to remember how many bytes to increment/decrement yourself.
Reading Knuth also helps with this. It's one of the reasons he uses a synthetic assembly language rather than a high-level language. There are no pointers. There is just memory, and you need some way to find stuff in memory, otherwise it is just noise.
Pointers are super hard if you've not dealt with indirection before. Especially in C/C++ when you can dereference pretty much anything and you get a runtime segfault with no other diagnostic. Do you not remember when that was mystifying? Given that a huge fraction of vulnerabilities are unsafe memory accesses, I think most people still make plenty of mistakes around pointers.
The analogy I like is the address of IHOP (International House of Pancakes). If I know the address of IHOP is 4567 Portobello Rd., why don't I have pancakes?!? It's the same with a pointer. Similarly, if I have a pancake, and someone calls and asks me for the address of IHOP, it's also wrong to bring the pancake to them in lieu of the address.
IMO I don't think the problem in C is pointers, it's that pointers are arrays, in the sense that they are syntactic sugar. This is incredibly confusing, and probably part of the reason why most modern systems languages wisely make arrays more e.g. {pointer, length}, sometimes introduce slices, where length is runtime, and make you do some syntactic work to pull out the pointer from the array.
Pointers aren't arrays because sizeof works differently and pointer arithmetic on a pointer-to-array doesn't work "right".
The more confusing part is the variety of ways pointers get used: allocated memory, opaque object handles, extra return values, modifiable input values, array indexing, function pointers, etc.
No, pointers are not arrays. Pointers can point to arrays and, because of the language rules, you can do pretty much anything you can do to an array through pointers, but they are definitively not arrays.
I think we see the problem here. I've been using C on and off (granted, never formally educated in it) for 20 years now and I don't know the difference.
An array is a variable that is declared to contain sequential data (using square brackets and the desired number of elements). It is immutable. A pointer is a mutable variable that can be made to point to anything, and it can be used to reference an array or any of its elements. These are different concepts, although the language allows a lot of operations in common between them.
I see. I mean, yes I obviously know the difference between pointers and arrays as general CS concepts (and articulated how other languages get it correct), but we're not talking about CS, we're talking about C. Perhaps I should have said in my original context, "array syntax" is the same as "pointer syntax". But while the CS concepts of "array" and "pointer" are indeed different, the C syntax elides the two. C does not track the immutable length of the array for you.
I think what you're getting at is that arrays are not really first-class objects in C. Rather it just gives you a bit of syntactic sugar to do the same thing that you could do with pointers anyway. But it's up to the programmer to complete the abstraction.
No, I described how it is in C. C has a proper array syntax, although a poor one. The problem is that pointers can do a lot of the same, so people get confused.
I do remember a time in my very early C career where I thought "pointers are arrays". But I'm not sure what caused me to think that. They are two very different things. A pointer is a memory address. An array is an area of contiguous memory containing a whole number multiple of some data type. Pointer arithmetic naturally supports arrays, but the pointer is not the array.
I could read and write commodore 128 assembler before I started to learn C, so I knew about adresses, adressing modes and how to use them. Despite that, I still had trouble wrapping my head around the concept of pointers at first.
There are multiple things that can make it confusing, and they kinda collude to make it harder to get.
In order to understand them at all, you have to know
- what a variable represents (a region of memory)
- what an adress/reference is (the index of said region in memory)
- how a value is stored in and read from said variable (by putting it into or reading from the memory beginnen at this adress)
- that you actually can treat an adress as a value, i.e store it in another variable
Thats already four deeply related, but still different things you have to wrap your head around.
To make matters worse, the way pointers often are used (and most of the time introduced), is to point to another variable. Now you have two adresses, and two values, one of wich is one of the adresses. If that sentence confuses you, that is how beginners feel about it as well!
Dereferencing is not straightforward either:
It makes a pointer behave as if it were the pointed to variable, which in the usual introductory toy examples makes it look like a totally unnecessary and useless operation.
Also the size of the pointed to variable is encoded via the pointers type, and does not have to match the size of the actually pointed to thing. Another mindblower that one.
Lastly there is the linguistic problem that we tend to say a variable IS its value. Which is actually misleading in multiple ways if a variable contains the adress of another one.
Ultimately I had to understand what pointers are NOT, in order to finally grasp what they are.
While there were macro assemblers, all I did use was the built-in machine monitor, which was basically a system provided debugger. But you could assemble single mnemonics into a memory location of your choosing, so I would put programs together directly in memory, one instruction at a time. That required counting their length as well, and doesn't support any constructs that span multiple instructions, like macros or variables...
I knew variables already from commodore Basic, which i learned first. But there was no concept of pointers, direct interaction with memory happened via peek and poke, and the adress space was so small, you just chose an adress where to put your stuff and worked with offsets.
It is worth remembering that on these old machines there was no OS managed memory allocation... So there was much less need to work with adresses unknown at "compile time".
Ah ok, I only got into Assembly when moving into 16 bit computers, on my Speccy I hardly did any Z80 coding, just a couple of Timex Basic stuff and mostly playing.
When I got started I found the syntax of it confusing, like why doesn't de-referencing look more distinct kind of thing. I got used to it eventually but it didn't feel very clear at first.
Same here. After I read K&R C's memory model seemed pretty simple and straightforward. Whenever I have trouble figuring out something I usually draw a little diagram with the memory space, where things are allocated and how handles and pointers relate to the memory space.
> Pointers are variables that hold a reference to something in memory.
This sentence is extremely confusing or meaningless. I would rather say: "A pointer is an integer that indicates a position in the RAM."
Thus, if a pointer "p" indicates a position in memory, then "p+1" is a pointer that indicates the next position in the memory. The value on that cell of memory is often written as p[1].
I disagree with this, and I think it's important to understanding pointers.
A pointer is a number. A variable stores a value.[1] A variable has an address, a value does not; a value might be stored at an address, but it doesn't intrinsically have an address. A person might live in a building, but the building has the address, not the person, who can move.
[1] (In C-like languages this is true. In Python, a variable stores something you're largely not allowed to inspect, which might be a pointer to a data structure with data and metadata. Haskell has no variables, just names for constants.)
If I put you in front of a wall full of numbers, and I tell you "point a number with your finger and remember it", what is your finger? A number? Or the thing you use to point a number?
yes, i find OP's definition very clear and by saying number instead of integer you made it even better.
for example i was re-reading some math stuff yesterday that made me realize that sometimes we want natural number instead of integer which can be negative...
Registers are simply the fastest, closest kind of memory for a CPU to use. They're still memory, in that you can put something there and remember it for later.
However they are implemented internally is a detail. Formally they are "affine" integers, in that you can compute the sum of an integer and a pointer to obtain another pointer, and compute the difference of two pointers to obtain an integer.
I agree that "a pointer is an integer" is an over-simplification. But it is a very useful one when you are learning C. Then you will have time later to enter into the gory details, once you fully master the simple case.
> and compute the difference of two pointers to obtain an integer.
This is only a legal operation if those two pointers are part of the same object / allocation. It's UB (see C11 6.5.6 Additive operators ; C++ is similar) to do for instance
Note that in both cases your undefined behavior can be defined if the set of valid pointers is contiguous and can be represented by the integers. In that case, the return value of ptr2 - ptr1 is the difference between the two integer representations, and ptr1 < ptr2 is true when ptr1 has a smaller integer representation than ptr2, and false otherwise.
So undefined behavior is not necessarily to be avoided. It needs to be evaluated on a platform-by-platform basis. It's not defined in the standard, because the standard is describing an abstract "computer" which supports the operations. However in the real world, with concrete computers such as x86 or ARM pointers have an integer representation and arithmetic and logical tests can reasonably expected to work, even if they have no standard definition in the abstract "C machine."
And it's tremendously unfair and probably a bug for any "optimizing compiler" to use that undefined behavior to do anything other than subtract the two pointers and return true or false when they're logically compared.
It might be true on X86 if you’re using a flat memory layout. With segmented memory or PAE, all bets are off. Thankfully PAE is usually the responsibility of the operating system which provides a flat address space to each process.
> And it's tremendously unfair and probably a bug for any "optimizing compiler" to use that undefined behavior to do anything other than subtract the two pointers and return true or false when they're logically compared.
They have to support arithmetic operations, however. Pointer offsets (p + 1) and pointer difference on the same object (p - q) should be well defined even if the pointer itself contains fruits and veggies.
That ends up being misleading though as it ignores optimizing compilers. A pointer isn't simply an integer address of memory. It's tagged (by the compiler) with information about e.g. which allocation unit it came from and what extent it has (i.e. how far you can move the pointer forwards or backwards).
The long and short of it is that doing `p+1` can be UB. Also two pointers can point to the same actual memory address but not be equal.
Pointers are, in practice, a complex abstraction. It can be hard to understand all the subtleties, especially if you're used to the simplicity of memory addresses in assembly.
It took me a minute to recall what you're referring to. To save everyone else the trouble: Pascal had pointers. And it also had var parameters, which are essentially pass-by-reference. (At least, I presume that's what pjmlp is, um, referring to...)
Not only, in Ada, Modulas and Oberons you can have them as local variables as well, and in the variants with GC support, they split further into traced and untraced references.
OK, first of all, the syntax and the semantics of a pointer and a reference are completely different from each other. Departing from C, one must start thinking of a pointer as a variable containing a value that is a reference, the latter now always being a constant. That's pretty subtle, I must say. Add to that a very important distinction between lvalue, rvalue, and "universal" references, and then one needs to stay very much current as a C++ coder to remember all that.
Some things should be left out until the kids grow up a little. (Otherwise you would have to describe gruesome details of swapping and memory mapping and how dereferencing a pointer might trigger reading from disk etc.)
I've always explained it to people like this: Imagine a big cabinet, like a bunch of safe-deposit boxes in a bank. Each box has an unique number written on it, and the boxes are simply numbered 0 to 99. Now when you open a box, you find a piece of paper in it with a number between 0 to 99. It tells you which box to open next - so it points to another box. That's a pointer.
Seems to work pretty well, at least for conveying the basic idea of pointers and dereferencing.
But why do you need to mention that this number is inside a box? It seems strangely contrived. Even if the number was outside the box it would be a pointer, it has nothing to do with being in the box.
Because usually a pointer is stored in memory in exactly the same way as the things it points to.
Another useful feature of this abstraction is that if you open a box and find a number you don't know whether it's a pointer to another box or whether the number means something else. This demonstrates how, at least in C, your types aren't stored in memory along with the values and you need something else to tell you how to interpret the value you find at an address.
When you explain floating point numbers you also put them into boxes? No need for that, you can explain them intrinsically, just like pointers.
Moreover, in practice, it is clear to think that you already have your pointer in your hands (e.g. in a register), no need for your pointer to come from the RAM that it is used refers to. This is an unnecessary recursivity.
>When you explain floating point numbers you also put them into boxes?
That makes no sense. If I was explaining floating point numbers with boxes, for whatever reason, I'd put the values inside the boxes, yes. The analogy just doesn't work here, but the analogy for memory makes perfect sense.
>This is an unnecessary recursivity.
No, it's not. Think of a linked list for example. Traversing it makes perfect sense with my analogy: You open the first box, look at the number that's inside, open the box corresponding to that number, look at what number is inside that box, open the next corresponding box, etc. - it all makes immediate and intuitive sense.
If you just start with "open box x" (because x you somehow already know magically because it's already in a register), you haven't really explained much.
Or think of pointers to pointers. With my analogy: Easy! It's just another box which contains the value of the next box. If this concept isn't explained, a pointer to a pointer makes no sense because the pointer value has no corresponding box, hence no address.
Seeing why pointers are used is especially important. Understanding that functions in C pass by value and what that means gives context to pointers as a useful tool instead of a longer and more complicated way to do things. Students become a lot more frustrated if they don't see a reason behind learning something, especially if there's any sense of elitism going on.
Additionally, a lot of these terms like that pointers now introduce (address, reference, etc) are being introduced without any sort of context, usually they're brand new to students. If you've never seen (and may never see) assembly then you miss out on a lot of what's going on at the beginning and eventually when you get it, as well. It's an abstracted view of what's going on.
I learned pointers by drawing boxes and arrows which I think is the best high-level explanation. Basically a variable is a container which holds a value from a predetermined set of values (also known as a type). For instance, an integer variable x holding the value 37 can be visualized as a box with the name x:
x
+--+
|37|
+--+
A pointer variable p can be visualized as a box which "contains" an arrow pointing to another variable of a certain type, for instance
Coming back to C after several years, I now think one of the hard things about pointers in C is the C syntax for pointers, not just the idea of pointers in general. Eg, sometimes the type declaration wraps around both sides of the variable name:
int a[2];
int (*ap)[2] = &a; // pointer to array of int
I agree that it is confusing. Here [1] is something that can help new C programmers to understand these kind of declarations. It gets easier after a while.
I feel like spending some time programming in assembler, even just Z80 or 6502, would give most people an understanding of pointers in very short order.
True, but you do not need to know assembler to have a general idea of what computer memory looks like; but more importantly, C's notion of a pointer is more complex as it is tied to the data type and its 'sizeof' and is further complicated by association with arrays.
Not a simple general intro to pointers
A useful gentle intro to the C type feature of pointers to arrays.
The C language has more than one way of accessing arrays. Other languages don't.
In the classic 80s teaching language Pascal if you started with pointers it was because you where working with linked lists.
Most people I have taught programming to conflated the pointer access in C and the linked list concepts to their disadvantage.
I taught linked lists first and left the point abstraction intact. Later I crossed the the layer of abstraction to explain memory access details.
I have had more success (less learning difficulties) this way.
> The C language has more than one way of accessing arrays. Other languages don't.
Like this?
procedure ArrayDemo;
var
data : array [1..20] of Integer;
ptr : ^Integer;
i : Integer;
begin
ptr := @data[low(data)];
for i:=low(data) to high(data) do
begin
WriteLn(ptr^);
ptr := ptr + Sizeof(Integer)
end
end;
To quote K & R "In C, there is a strong relationship between pointers and arrays, strong enough that pointers
and arrays should be discussed simultaneously" ... however they also say "Pointers have been lumped with the goto statement as a marvelous way to create impossible to understand programs". I use both these quotes in my lectures.
I remember pointers being very hard to grok when I was first learning about them. And then once I was finally starting to understand, I saw a char and my head exploded. It took some time & practice for it to become second nature, but eventually it did.
It's interesting how most languages have this concept, even if they don't call it "pointers". In some ways that reduces complexity and learning curve, but in other ways it leaves invisible footguns lying around.
JavaScript comes to mind as one where references and referential equality are very important to understand, especially in modern frontend dev with its focus on immutability, but there's no syntax like C's `*` and `&` to lend clues about what's going on. I ended up writing a visual guide to "pointers for JS devs" [0] to help explain the concept, because I think it's really easy to get bitten by reference bugs when your mental model doesn't match what's happening.
I wish someone would do a gentle introduction of header files. I sort of understand pointers, at least on a basic level. Headers files are apparently much easier, so they get the briefest of introduction in most "Learn C" tutorials and books, but I don't understand how they work or how to use them.
There is no magic in .h files, you can think them as plaintext. The actual magic happens in .c files. Since C requires objects to be declared before using them, you need to declare objects used in multiple units together. Let's say you have a function `int f(int)`. You want to use this function both in apple.c and pear.c. How would you do it. Well you need to write a f.h such as
int f(int);
then in apple.c
#include "f.h"
int apple() {
int x = f(10);
// rest of the code
}
You can think this as a copy paste of f.h such that, apple.c is equivalent to:
int f(int);
int apple() {
int x = f(10);
// rest of the code
}
but what "is" f. To "define" f you need another .cpp file that implements f, let's call it f.c:
#include "f.h"
int f(int x) {
return x*x;
}
now you compile f.c, apple.c, pear.c. These will create f.o, apple.o and pear.o object files. Then you link these together to create your application. e.g. if f.h is in include/ dir:
this is an error because the compiler treats each .c file separately, so when it’s compiling main.c, it isn’t looking at f.c, so it has no way to know the function f exists. So you need to “declare” it at the top of main.c. A declaration is similar to a definition, except it has no code, and just tells the compiler that the function exists:
void f();
But now imagine f.c had many functions. It would be extremely cumbersome to repeat all those declarations in every file that wants to use them. So you can collect all the declarations in some file f.h, and include it from main.c or anywhere else you want to use functions from f.c. When the compiler sees #include "f.h", it operates as if the contents of f.h were copied and pasted at that point.
You might ask “why not just include f.c from main.c instead?” Because then you would have two _definitions_ of f in your program (one in f.c and one in main.c), which is illegal, as opposed to two declarations, which is fine.
> this is an error because the compiler treats each .c file separately, so when it’s compiling main.c, it isn’t looking at f.c, so it has no way to know the function f exists.
But it's not an error and the compiler does not need to know the function exists. If there's no declaration, then the function's type is implicit and simply assumed to exist.
If the function doesn't actually exist, then it's your linker that will complain.
You're right (even with `gcc --std=c99 --pedantic`, which surprised me!) I should have said that the compiler won't be able to type-check the arguments and return value.
Another good trick for seeing what headers (and the pre-processor) does in general is using the -E flag (for gcc / clang) it will show the file with the headers included so you can see what happens. This is really useful when using macros and include guards to see what you are getting. clang -E myfile.c > pre.txt will give you an editable file to see what is going on.
What I find most annoying about tutorials on pointers (as someone relatively new to C picking it up for work and fun) and memory management is that every single one of them seems to discuss the trivial case of a hard coded char or int array.
Which like, is great for the absolute basics, but not for just about every serious piece of software where in practice what you need is to build stuff like e.g. dynamic accumulators of strings -- and virtually no tutorials or books I've read talk through best practices for doing this.
So I've sort of kludged together my own probably wrong set of patterns for this. Reading the source code of others on github helps a bit but I have yet to find a good example of how to e.g. read strings from say a CSV file and parse it into an array of arrays of strings
Perhaps that's because C pointers are usually (always?) exactly that - a hard coded char array, which starts at memory offset 0.
Depending on the circumstances, it may be easier to think about it as int array, or some other type, but the easiest on-ramp for pointers is still char[] = 0x0.
> how to e.g. read strings from say a CSV file and parse it into an array of arrays of strings
>
> Does anyone have any pointers here? (sorry)
I think this kind of thing isn't discussed too often because there are so many different choices that you can make (and once you understand the language & your constraints, you can certainly make those choices). I agree it would be useful to hilight all the possibilities. Let me try to illuminate:
First, do you really need an array of arrays of strings? That's possible, but not all that common. If you don't really need such a thing, then you do something like strtok or strsep: implement a function that you call in a loop to extract fields one by one. Now you just need a single buffer large enough to hold a single field (or one row if you prefer to make it a little simpler). Your function locates the terminating comma, replaces it with a NUL, and returns a pointer to the start of the buffer (and stores the start of the next field -- that is, the address of the byte after the NUL -- somewhere for the next call).
This approach is popular among C programmers because it usually means you don't need dynamic allocation, as long as there's a reasonable maximum field size. Or if a maximum field size cannot be imposed, then you get away with dynamic allocation (and re-allocation, when needed to grow) of a single buffer.
What next? Ok, maybe you really do require "all fields at once." Your average C programmer still won't allocate arrays of arrays of strings. Instead, they read the entire row, extract fields out of it (as above), and store the pointers to the start of each field somewhere. If the caller knows how many fields they want to deal with (very often you do!), then they can provide you a fixed length array for these pointers. Otherwise, you can dynamically allocate a buffer to hold the pointers. Either way, now you need two buffers (and depending on your needs, one or both or neither may be statically sized): one for the row, one for the pointers. Now the buffer containing pointers is exactly like argv in main(), while the fields are in one contiguous buffer spliced by NUL bytes.
Ok, maybe you really do require all rows and all fields at once. At this point C programmers will hate you because you're forcing them into dynamic allocation (or an unreasonable fixed size limit on the file). Otherwise, you do as above, but you also terminate lines with NULs and now you have some choices as to how to lay out your row + field pointers. For example, you could allocate one array for row pointers, which point to field pointers (which could be allocated separately for each row, or, perhaps preferraby, in a single flat array containing pointers to all the fields). If you expect the caller to iterate (rather than random-access) rows, you could use a single mixed type array containing row metadata + field pointers.
Another very common approach would be to allocate field pointers + data for each row separately and then link rows in a linked list. Again this favors iteration but you're less likely to need to resize huge arrays.
Sometimes you deal with memory that is best to keep immutable; in that case, splitting the data into rows and fields with NUL bytes is not an option and you must take a different approach. Either duplicate the data (more mallocs, required if you also need to do things like unescaping) or keep it where it is and store pointer + length tuples.
See, there are lots of choices, and lots of variables, depending on how much memory you want to use, can you use fixed size buffers, what kind of access pattern the caller expects, does the caller expect to be able to free individual rows, etcetra.
There is no best practice but generally C programmers gravitate towards using iteration & fixed size buffers unless more is required (this is simple and lets the caller persist data if they want to, but doesn't force them to deal with dynamic memory if they don't want or need to). If something more specific is required, then you'll know what is required and implement something that supports just that.
The Python style divide-and-conquer where you first read an entire file, then chop it into lines, then chop lines into columns, etc. is not very popular.
I think people get scared of pointers because the numbers are big and are in hexadecimal. But they don't have to be. I didn't understand pointers until I thought of them as house numbers in postal addresses. In a world where everyone lives on the same street, you can send anyone a letter by mailing a letter with their house number written on the envelope. You don't even need to know their name -- thus SPAM mail is born. The same analogy holds for phone numbers. The pointer is like a phone number or house number. The person on the other end is the value you get when you dereference a pointer.
The point where I actually understood everything about pointers was when i realized that they just hold memory address. After that everything like pointer arithmetic and dereferencing pointers become clear as day.
It is easy to believe that you understand something while in reality things are more complicated. What is an address? Is it like an integer value? (Is it really? Or can it be converted to one? Even if so, how many bits wide is it?) Or, if not, what is its structure? Back in the day, a question would be is it a 'far' or a 'near' pointer or we are living in a flat address space? Also, how is pointer arithmetic affected by alignment?
Yeah, you may use a pointer in C, but it will compile to an instruction that fetches some data from a location in memory relative to the currently executed instruction.
The fact that in C pointers point to specifically objects and not just simply to a random place in memory makes all the difference and is source of oh so many bugs
But I hear you say: can't an int or long also do that? All you need is the number of the memory address.
Well, the magical thing about a pointer is that it can use a special operator!
What? (I hear you say)
Yes!
Here is what it does: it allows you to obtain the value that is stored at that memory address.
What does a pointer store again? (a memory address)
But it can also obtain a value at that memory address.
You can't do that with an int, I've tried [0].
We call this operator the dereference operator [1].
It looks like a star
*
That's the one. That's the dereference operator.
One of the confusing things is that when you declare a pointer, you also use a *
Example
int *a;
//or
int * a;
//or
int* a;
In those 3 examples, the * is not a dereference operator. It simply indicates that it is a pointer. In this case we have an int pointer.
I always like to view it as: we have an int! And this is how many times we're allowed to use the dereference operator: one time
How many times are you allowed to use it on this pointer?
int *****super_pointer
That's right, five times!
So when you have
int val = 10;
int *a = val;
Then what is the value of a? It's 10. What do pointers store? Addresses. So what address is this? Address 10!
But when it comes to memory addresses in computers, there is a tricky part to it. In normal operating systems, low addresses are reserved for special things, I think it's the kernel in this case. So if you dereference that particular address, the OS/runtime environment won't allow you to. They will give you an error called a segmentation fault.
So you need to get a valid address of the variable val, but how do you do that? We have another operator for that, which I like to call the address-of operator.
So when you have
int val = 10;
int *a = &val; // & is the address-of operator
Then the variable a has the valid memory address we want!
There's much more to learn, but I'll leave it at this for now.
---
I recommend people who start learning pointers/programming to use GDB with the TUI graphical interface. One could also print the values, but it's easier to see what really happens. For that there are two steps you need to do:
> In those 3 examples, the * is not a dereference operator, it simply indicates that it is a pointer
This is one of the confusing parts for beginners, but in actuality, C declarations are meant to mirror the use of the variable. In that sense, you can think of the asterisk in the declaration as the same unary operator:
And this is how many times we're allowed to use the dereference operator: one time
I can see what you meant, but I initially thought this was trying to say that a pointer declared with a single asterisk could only be dereferenced once and never dereferenced again afterwards (as if there was a "one-time-use only" property to pointers). In the same vein, the "super_pointer" example could be misinterpreted like "Oh, so if I want a pointer that will let me dereference it on 5 different occasions, then I'll need to declare it with 5 asterisks?" I think a better way would be to replace the "how many times we're allowed to use the dereference operator" explanation and start with explaining the idea of a double pointer
int **doublePtr
in depth first. With a solid understanding of a double pointer (an address that points to a memory location that contains an address that points to another memory location), making the mental jump to understand
int *****super_pointer
should be easier.
Also, I think low addresses aren't used for various reasons, but I believe the kernel is stored in the top, high-address region of the address space. Otherwise, I think this is a really solid explanation of pointers!
> I can see what you meant, but I initially thought this was trying to say that a pointer declared with a single asterisk could only be dereferenced once and never dereferenced again afterwards
Ah, haha, I see! You're right. This could've been me back in the day.
> Also, I think low addresses aren't used for various reasons, but I believe the kernel is stored in the top, high-address region of the address space.
This is similar to the idea that a number isn't always a number. A number can be categorical, in which case comparison operators don't make any sense. They can be a label or a name, in which case addition or other mathematical operators don't make any sense. The same way a number can be an address, which has its own sets of operations that operate on the idea of an address.
As an example, statistics has the idea of ordinal values vs interval values, where they have different properties that make them not comparable to each other and shouldn't be mistakenly be confused when working on them.
Haha, I do view mathematics as "software engineering but then purely for numbers".
I view things like the quadratic formula as little software programs. I view complex numbers as the necessary "software system" to allow for negative square roots.
Other views I have on mathematics look at this perspective and shout "blasphemy! How could you insult such an intellectual pursuit!". But it works quite well for in me when I'm doing math.
I like your lesson. I never saw it that clearly. And with respect to numbers, I don't think I saw this at all.
I would recommend working with https://godbolt.org instead of gdb. It abstracts away the unnecessary stuff and highlights lines corresponding to asm from source and vice versa quite conveniently
Yea, I agree. That's why I said there's a lot to learn. Especially also when I think about how things bind, and so on.
When I look at a statement like this, I really have to sit for it and just reason though it a few times.
You might be able to tell: I'm not a C programmer ;-)
I simply self-taught C in order to get some homework assignments done with some security classes. Example: we had to make our own packet sniffer in C. Actually, we had to make all kinds of tools in C.
as a C noob, this example is humorously confusing...
declare an array, fair enough, makes sense.
Declare a char *pointer,(a "reference to something in memory") ok if you say so.
Assign the char pointer the ARRAY name.... erm, ok.. so ... that would make the char pointer the 'first char in the array' then?
lol no you silly old duffer, it sets the char pointer to the memory address of the first char in the array.. not the contents of that location, obviously ;)
And why is that? I am a certified C noob and this really confuses me. Why `alphabet_pointer = alphabet;` points exactly to the first element in the array? Why not the whole array? Why not the last? Can we have a pointer that accepts a whole array?
Because most of the time* an array name decays to a pointer to the first element of the array.
So when the array is declared
char alphabet[10] = {'a', 'b', 'c', 'd', 'e'};
What the alphabet variable "holds" can be seen (this is not exactly true) as a pointer to the array.
Then when you do
alphabet_pointer = alphabet;
You just assign to alphabet_pointer the position in memory of the (first element of) alphabet.
Then alphabet_pointer can be dereferenced to access the content of the array (and not the address of a pointer to the array).
__
* There are some situations where an array name is not considered as a pointer to the array. Notably &array gives you back the address of the array : &array == array.
> Why `alphabet_pointer = alphabet;` points exactly to the first element in the array? Why not the whole array?
Someone decided it has to be so, because it is pretty convenient. Usually when dealing with an array, you want to do things with its elements and therefore it's extremely convenient that an expression with array type is converted to a pointer to the array's first element. If that didn't happen, then you'd very often have one extra layer of annoyance to go through in order to access array elements.
> Why not the last?
The first element is convenient because then you can reach for the other elements by adding a zero-based offset or index to the pointer. How often do you operate on an array starting from its end? How often do you like to work with negative indices? That's why not the last.
> Can we have a pointer that accepts a whole array?
We can have a pointer that points to a whole array:
int a[50];
int (*p)[50] = &a;
printf("%zu %zu %zu\n", sizeof a, sizeof a / sizeof *a, sizeof *a);
printf("%zu %zu %zu\n", sizeof *p, sizeof *p / sizeof **p, sizeof **p);
> 200 50 4
> 200 50 4
But what is an array anyway? It's just a bunch of (contiguous) memory. That bunch of memory has to start somewhere.
Since an array is just a bunch of memory, by pointing to the beginning of that bunch of memory you are pointing to the entire array.
Here follows a more complicated version:
What the tutorial is not telling you (and now you will hate me for doing things more complicated) is that in C, the alphabet variable is (or can be) treated as a pointer.
If you print the value of alphabet as an number (casting it to unsigned int, for example), you will see that is a position in RAM. That position is the beginning of the array. When you do `alphabet_pointer = alphabet;` you assign to alphabet_pointer the value of alphabet.
If alphabet array starts at address 0x1234, then basically you are doing
alphabet_pointer = (char*) 0x1234;
Also note that doing `alphabet_pointer = alphabet;` is the same as doing `alphabet_pointer = &alphabet[0];`, being alphabet[0] the first element in the array.
a c array is literally just a sequence of objects in memory (possibly with padding, but you can ignore that for a while). you can think of pointing "exactly" to the first element as being equivalent to pointing to the whole array. you need to know where the array begins and its size to do anything with it. but once you know where it starts, you can access the next element by adding sizeof(char) (or whatever the element type happens to be) to the pointer.
If you declare a structure of unaligned size, the compiler might (probably must) introduce some padding bytes at the end of the structure to fit the alignment. In some architectures you cannot do unaligned accesses (ARM for example).
Unless you pack the structure with the alignment you want.
Yes, so the structure itself has padding (added by the compiler). But that is regardless of whether it is in an array or not. OP seemed to suggest that C arrays introduce padding of some sort, which they do not.
It's long been the convention here, and it's useful for people to know how recently it's been published.
With an article title like this, it could easily be anything from 1 to 30 or more years old, and showing the date makes it easier for readers to know if they've already seen it before clicking through.
If pointers are so "simple", why are they so confusing?!? Because pointers are one of those concepts that's easier to use than to explain.
Some concepts like pointers (and also monads and also English words like "a" and "the") are easier to teach by examples instead of by definitions.
E.g. Consider the English words "a" and "the": a 5-year old can fluidly use those connecting words when talking but a 50-year old adult that's been speaking English for decades will have a tough time defining what the words "a" and "the" _mean_ without giving a confusing and circular explanation. Attempted explanation: the "a" is the indefinite or general signifier while "the" is the definite or specific signifier -- what's the problem?[1]
How did small children learn how "a" and "the" work without being given a formal definition? Because of numerous examples.
By working through enough examples, the lightbulb moment happens ("oh gee, is that all pointers is?!?") and the learner is able to derive his own definition. But the irony is that the learner now has The Curse of Knowledge[2] and he/she now thinks he can write a "better" tutorial to explain pointers but new learners reading that tutorial will still say "I don't understand your explanation at all."
[1] riff on : https://stackoverflow.com/questions/3870088/a-monad-is-just-...
[2] https://en.wikipedia.org/wiki/Curse_of_knowledge