There are only two types of pilots: 1) Those who have heard the magic words from ATC: "When you land, call this number…"; and 2) those who someday will.
What is ATC Really Up To?
Sometimes, it's obvious to a pilot why they were asked to call. You were assigned: "Climb and maintain 5,000 feet", but you went to 5,700 feet and ATC tells you to get back down to your assigned altitude. On other occasions, it's not so obvious. You may have scraped the edge of a Bravo or Restricted Area and didn't know it. In any event, if ATC learns of certain Possible Pilot Deviations, they are required to start an inquiry and collect data. If time permits, ATC is also required to advise the pilot of the possible deviation, also known as a "Brasher Warning" (see FAA Order JO 7210.632)
The first question is: should you call? The answer is: it depends on how strong and disciplined you are.
If every pilot were a robot, they could be preprogrammed to call ATC, provide only their name, tail number, pilot certificate number, confirm who was PIC, and ask what the nature of the situation is. ATC would give a brief description of the event, and afterward the pilot either would or would not get a Letter of Investigation from the FSDO. Unfortunately, we are not robots. A safety conscious pilot takes pride in doing everything right, and if something wasn't done right, we feel guilty. That guilt often translates into a really bad phone call with ATC that starts out with the pilot confessing something like: "I'm really sorry for what happened, I wasn't paying close attention to my assigned altitude." One small problem… on this call ATC wasn't concerned with any altitude deviations. The pilot may have been asked about an entirely different matter, and now the pilot has just confessed to a violation that ATC didn't know about.
Bottom line: unless a pilot knows what to say (or not to say), it's often better to call an attorney first to talk through the situation first. Afterward, the attorney may recommend you call together, or in some cases, not call. If you decide to call, be polite, be humble, and be gone. A phone call with ATC that lasts more than two minutes is too long. Do not make statements or give explanations. They can and will be used against you.
What happens if you don't call? Well, it depends.
Some violations (such as certain altitude deviations that causes a loss of separation) must be sent up the chain to the FSDO (called a Mandatory Occurrence Report), so calling ATC immediately won't help much. If you don't call, there is zero penalty for doing so. If the FAA wants to bring an enforcement action, the FAA generally needs to do it with evidence independent of the pilot's statements. Further, the lack of cooperation or statements by the pilot is not admissible as evidence of guilt in an NTSB hearing.
Other situations are discretionary, and a phone call might nip the matter in the bud. Back in 1991, I was flying out of Grand Forks, ND in a Piper Tomahawk with the DG pulled out for maintenance. I was cleared for takeoff on 35L (midfield with about 4,000' out of 7,350' available), My brain short-circuited, and the DG wasn't there to save me. I took off on 17L instead (same pavement, wrong direction). Tower told me immediately after takeoff what I did wrong (duh), and asked me to call when I landed, which I did. I apologized profusely for taking off in the wrong direction, explained that my DG was pulled out for maintenance, and that I felt terrible. I ended up getting a free tour of the control tower, shaking hands with the boys upstairs, and leaving with a sigh of relief. Not everyone has been so lucky.
FSDO Letters: Call us in 10 Days, Please
Sometimes, the decision to call or not to call this number isn't a request from ATC, but rather in response to a letter from the local FSDO. A couple of years ago I represented a pilot who had a gear up landing in SoCal. Usually, these matters are investigated, and absent unusual circumstances, the pilot is asked to undergo a re-examination of certain skills (called a 709 ride) instead of an enforcement action. Unfortunately, when this pilot got the FSDO's letter, he called and freely answered every question the Aviation Safety Inspector asked him (they read off of a checklist from their Form 8020-23, or ATQA). This inspector went deeper, asking: "How were you feeling on the date of the incident?" The pilot replied, "Not great, I was getting over a cold". The ASI's next logical question was: "Were you taking any medications at the time?" The pilot replied: "Yeah, Nyquil, but that was the night before". Unknown to this pilot, Nyquil is not one of those 8 hours bottle-to-throttle drinks. It has a recommended 60 hour waiting period before flying, and the pilot was busted on his voluntary statements alone.
The decision to call (or not to call) ATC or the FSDO is a variable one, with no standard answer. If you do call, know what to say and what not to say. If a pilot is the least bit unsure, it's better to consult with an attorney before making the call.