Creating a Simple Golang Network Application, Part 3

In the final version of the utility I want port_listen to keep running until the user tells it to quit. A simple way to do that is to wait for a keyword to be entered.

First I need to import a couple of additional libraries.

import "os"
import "strings"

Next in my opening block of variable creations in the top of main() I add a control variable using "var strUserInput string" between the declaration of i and the channel, then, still inside main():

for {
 fmt.Scanf("%s", &strUserInput)
 strUserInput = strings.ToLower(strUserInput)
 if strUserInput == "quit" {
  os.Exit(0)
 }
}

The "for" creates an infinite loop...at this point you can kind of tell infinite loops can be handy. fmt.Scanf is looking for using input, which is stored in strUserInput; it's passed by reference because of the way arguments are passed in Go functions. Next I use the ToLower function from the strings library to lowercase the entered text before the "if" statement performs a comparison.

I thought of the lowercase transformation after I wrote the function to initially look for the word "Quit"; I capitalized it out of habit. When I ran the application and tried using "quit" it of course wouldn't work. Then I realized my mistake. Whoopsie!

Transforming the string entered by the user to all lowercase was a quick and easy way to change the recognized keyword from one to several variations of the word "quit." When it's that easy...just do it.

The last two lines compares the strUserInput contents to the word "quit" and if it matches call the operating system's "Exit" routine passing the exit code status of 0 (which normally means the application is exiting normally.)

At this point I felt the mini-application was shaping up nicely. Next task to hammer on: the network code.

ListenToPort() was the launching point for the port monitoring, so it was the first to get a revamp.

In the initial source I had written:

func ListenToPort(uint16Port uint16, chanOutGoing chan<- string) {
 var strDiagMessage string

 strDiagMessage = "Listening to port " + strconv.FormatInt(int64(uint16Port),10)
 LogEvent(strconv.FormatInt(int64(uint16Port),10), strDiagMessage)
 chanOutGoing <-"done " + strconv.FormatInt(int64(uint16Port),10)
}

Glancing over the function you can see it does little more than acknowledge that it's listening to a port (obviously it wasn't, but it was a placeholder for that code) by printing a string for the user followed by a quick blurb back to main() through a channel.

The revamped code was a wee bit more complicated.

func ListenToPort(uint16Port uint16, chanOutGoing chan<- string) {

 var strPortToListenOn string
 var strRemoteAddr string
 var strRemoteAddrSansPort string
 
 strPortToListenOn = strconv.FormatInt(int64(uint16Port),10)
 
 netListenFor, err := net.Listen("tcp",":" + strPortToListenOn)
 if err != nil {
  LogEvent(strPortToListenOn, err.Error())
  chanOutGoing <-"done" + strconv.FormatInt(int64(uint16Port),10)
  return
 }
 chanOutGoing <-"done " + strconv.FormatInt(int64(uint16Port),10) 
 defer netListenFor.Close()
 
 for {
 
  conn, err := netListenFor.Accept()
  if err != nil {
   LogEvent(strPortToListenOn, err.Error())
  }

  strRemoteAddr = conn.RemoteAddr().String()
  strRemoteAddrSansPort, _, _ = net.SplitHostPort(strRemoteAddr)
  LogEvent(strPortToListenOn, "Connection attempt made from " + strRemoteAddrSansPort)

  go GrabInput(conn, strPortToListenOn)
 }
 
}

The diagnostic string declaration is gone and in its place are string variables for the port to listen on and handle the remote address. I'm certain there's a way to shorten the code or refactor it so I didn't need to break out the remote address bit into multiple strings, but I felt this was a little more explicit in what operations were being performed.

The strPortToListenOn variable, as you can tell from my naming convention as well as the declaration, is a string. The next line converts the port...passed in the argument list as uint16Port meaning it's an unsigned 16 bit int...into a string by calling strconv's FormatInt function. Because that function needs a 64 bit integer I have to cast uint16Port as a 64 bit integer and tell it to use base 10 (that's the second argument passed to FormatInt) in the conversion function.

I know, it's strange and seems wasteful but I didn't see a straightforward 16 bit conversion function in the documentation so I just cast it as 64 bit integer.

Next in the execution flow is the network listener. netListenFor is a listener returned by net.Listen with the type tcp and the port assigned with the value contained in strPortToListenOn. To be more accurate, it listens on ":<strPortToListenOn>" as the empty part prefacing the colon means the listener should listen on all local network interfaces. That way multiple interfaces, like wireless and wired ethernet, can be monitored.

Following the attempt to create a listener the value of err is evaluated to see if something went wrong. Go is very big on error checking; it's idiomatic to check immediately after function calls to reduce bugs. If an error is encountered (meaning there's a value in err, making it something other than nil) the code calls LogEvent (gotta tell the user something went wrong...) with the port number (so we know which go routine had the problem) and the error message. It then executes "return", effectively breaking out of the go routine. Most common reason something would "error" here is another process is already listening on that port; the return means that goroutine stops running, but the program will keep executing.

There is one line of code that is repeated in both the error block and immediately outside the error block, and that's a "done" string sent to the channel. This is still part of my proto-framework; in main() the channel is being monitored to tally that all the ports are being monitored (or trying to be monitored.)

In the previous form the same thing happened, only each ListenToPort goroutine only made one call. This revamped version also makes one channel communication but it's listed twice because if the attempt to listen fails the second channel communication never happens; if the attempt to listen succeeds the err check never runs. If even one error occurred and that second channel reply wasn't there the tally in main() will never reach the full count and the application would get "stuck" in an infinite loop waiting for the missing, errored connections to report back.

The deferred Close() function means the application should keep the port listener open until the function returns. This lets the listener get handed off and get cleaned up automatically when the function is finished doing what needs to be done (and reduces bugs introduced when the programmer forgets to close handles/connections! Defer is handy when dealing with sockets and files!)

The next for{} block is the bit that handles actual connections. A connection is listened for and the accept of the connection is returned as a connection type to the variable "conn", along with an err message if there's an error. Again, if there's an error, it's evaluated by checking whether err is nil and if it isn't a message is sent with the port number (thus identifying the problem port "process") and the contents of the error message.

Then I wanted the IP of the machine that was making the connection, because knowing the (supposed) origin of the connection attempt would be great for troubleshooting ("What's connecting to me?"). I started by assigning strRemoteAddr the return value of conn.RemoteAddr().String(); it was very handy that RemoteAddr has the ability to return a string value!

The problem is that this contains the IP octals along with the random port to which the connection is handed off (A connection may initiate on a known port, but then often gets handed off to an unprivileged random port in the high range). I wasn't interested in that; I just wanted the IP address. So I next assigned the strRemoteAddrSansPort the first return value (the other two going to the _ (underscore) meaning we're not interested in the return values there...toss'em!) of the net library's SplitHostPort function. At that point I can send a log message containing the port number (the goroutine's "id", so to speak...see the pattern?) and a string saying there has been a connection attempt from <IP value>.

At this point the flow of control is handed off to a new function called "GrabInput" with the connection (called conn) and the strPortToListenOn as arguments. The for{} loop then continues a new iteration (remember, the go keyword says GrabInput should spin off to do its thing asynchronously.) That means if a connection is made on port 123, GrabInput will grab that connection to do its thing and a new Listen is connected to port 123 to wait for a new connection. If multiple attempts are made to hit port 123 they'll all get handled; if I didn't do this, connections would be serialized.

Quick story; I experimented with a web server (very very very basic version) in Go using the online tutorials. I discovered that if you didn't properly handle the incoming connections as described in the previous paragraphs, what happens is you can connect and get a web page, but until the web browser was done and closed the socket nothing else could get a web page. That first page serve was snappy, though.

Back to the topic at hand...ListenToPort is done! Now on to GrabInput(), a new addition to the program. Here's that function block:

func GrabInput(conn net.Conn, strFromListenerNumber string) {

 var strMessage string
 var strRemoteAddr string
 var strRemoteAddrSansPort string
 
 bufIncoming := make([]byte, 1024)

 strRemoteAddr = conn.RemoteAddr().String()
 strRemoteAddrSansPort, _, _ = net.SplitHostPort(strRemoteAddr)
 
 for {
  bytesRead, err := conn.Read(bufIncoming)
  if err != nil {
   LogEvent(strFromListenerNumber, err.Error())
   return
  }
 
  strMessage = string(bufIncoming[0:bytesRead-1])
  LogEvent(strFromListenerNumber + ": Remote IP " + strRemoteAddrSansPort, strMessage)
 } 
}

GrabInput takes the network connection and port being listened on (the former as a connection type and the latter as a string type) as arguments. In the beginning I declare three variables, basically duplicating some of the code from ListenToPort, creating string variables for strMessage, strRemoteAddr, and strRemote AddrSansPort.

Next comes the heart of the function; a buffer is created called bufIncoming with a length of 1024 bytes. Pedantically this is a buffer in how it's being used...it's using syntax to create a Go slice, of type byte, with a length of 1,024 and returns the slice to bufIncoming. Basically a roughly one kilobyte "buffer", for practical purposes. (Note: I need to get this clarified, as I may be mixing up what is happening here...I know the effect of what is happening, but is it a slice? An array? Pedantically not a buffer?)

The next couple of lines duplicates what was done previously in ListenToPort; it extracts a string with the IP address of the connection.

The first line of the for{} block reads the information coming into the network connection (conn) using the Read function (conn.Read) and puts the contents into bufIncoming while simultaneously getting the number of bytes being used (the code creates and assigns the returned integer...the count of elements in the slice used...into byteRead; you can see the aggregation of creation of the variable and the assignment being performed because of the use of the colon and equal sign.

At this point you have a count of the number of bytes used in the slice and the actual information that was sent to the socket contained in bufIncoming. And an error message, if one occurred, assigned to err. If there is no data in the buffer, the information read is EOF (end of file.)

As is typical in Go applications the next bit is the checking for an error...if err is not nil, send the error message (in this program it's the port number as a string...again...along with the error message itself) sent to the LogEvent function.

The next bit takes the strMessage variable and assigns the contents from bufIncoming. This is done by using the string function to pull the contents from bufIncoming, reading from item 0 to one short of the length of the information. Why? Lengths and counting starting at 0 must be accounted for or you'll get an out of range error (and in Go, a crash.)

That message is then once again passed to LogEvent with the port number and a message containing the remote IP address and the contents of what the remote client sent.

At this point the bulk of the networking portion is completed; there are a few aesthetic changes made to clean up a little. For example, the original LogEvent said:

fmt.Println("From " + strFrom + ": " + strMessage)

But now it says

fmt.Println("From port " + strFrom + ": " + strMessage)

...so I didn't have to keep including "port " with every call to LogEvent. I also removed the line in main() that printed a diagnostic message whenever a ListenToPort process sent a "done" message through the channel; it just printed a thousand-number counter. The iterations are still counted, it just doesn't announce it now.

I added a "Waiting for Quit" line in main() so the user would have a prompt that the program could be exited by typing Quit, making the application slightly more user-friendly.

I used "go build" to create a new binary and then ran a test of the latest version; this uncovered a new set of complications...

First, ports below 1024 are considered privileged in Unix, and only a privileged user can open those ports for monitoring. The previous test versions of the code would just run. Now it only runs if executed using sudo. Well...that's simple enough to fix. Use sudo to invoke port_listen.

The next item I bumped into were limits built into the shell and kernel for user processes. A certain number of ports would open for monitoring before a slew of "too many open files" would pour into the terminal, which I believe is triggered because Unix treats the open ports as file handles. The way around it was to run the following two commands:

sudo launchctl limit maxfiles 1000000 1000000
ulimit -n 1400

Then I could listen to all the ports. I haven't tested yet if multiple interfaces would surpass the limit, but I don't think so since with just the wired ethernet connection the application is listening to 2,048 ports (the wired IP address ports plus all the localhost ports,) surpassing the 1400 limit imposed by the shell after modification.

These changes are temporary; a new shell or reboot will have the previous shell limits imposed again. To get around them permanently would mean some alterations to configuration files or automating some method of altering these limits when executing port_listen...

So where do we stand with the state of the application? Port_listen now launches and listens all ports 1 through 1024 and prints to the standard output the content sent by remote connections, and exits when the user types "quit" and hits enter. Here's the source code:

package main

import "fmt"
import "strconv"
import "os"
import "strings"
import "net"

func main() {
// Opening so many ports brings a "too many open files" error. The way around it in
// OS X: sudo launchctl limit maxfiles 1000000 1000000; ulimit -n 1400; sudo ./port_listen
 var i uint16
 var strUserInput string
 chanCommLine := make(chan string)
 
 for i = 1; i < 1025; i++ {
  go ListenToPort(i, chanCommLine)
 }
 i = 0
 for {
  strChanData:=<-chanCommLine
  if strChanData != "" {
   i++
  }
  if i == 1024 {
   fmt.Println("All channels accounted for")
   break
  }
 }
 for {
  fmt.Println("Waiting for Quit...")
  fmt.Scanf("%s", &strUserInput)
  strUserInput = strings.ToLower(strUserInput)
  if strUserInput == "quit" {
   os.Exit(0)
  }
 }
}

func ListenToPort(uint16Port uint16, chanOutGoing chan<- string) {

 var strPortToListenOn string
 var strRemoteAddr string
 var strRemoteAddrSansPort string
 
 strPortToListenOn = strconv.FormatInt(int64(uint16Port),10)
 
 netListenFor, err := net.Listen("tcp",":" + strPortToListenOn)
 if err != nil {
  LogEvent(strPortToListenOn, err.Error())
  chanOutGoing <-"done" + strconv.FormatInt(int64(uint16Port),10)
  return
 }
 chanOutGoing <-"done" + strconv.FormatInt(int64(uint16Port),10) 
 defer netListenFor.Close()
 for {
  conn, err := netListenFor.Accept()
  if err != nil {
   LogEvent(strPortToListenOn, err.Error())
  }
  strRemoteAddr = conn.RemoteAddr().String()
  strRemoteAddrSansPort, _, _ = net.SplitHostPort(strRemoteAddr)
  LogEvent(strPortToListenOn, "Connection attempt made from " + strRemoteAddrSansPort)

  go GrabInput(conn, strPortToListenOn)
 } 
}
func GrabInput(conn net.Conn, strFromListenerNumber string) {
 var strMessage string
 var strRemoteAddr string
 var strRemoteAddrSansPort string
 
 bufIncoming := make([]byte, 1024)

 strRemoteAddr = conn.RemoteAddr().String()
 strRemoteAddrSansPort, _, _ = net.SplitHostPort(strRemoteAddr)
 
 for {
  bytesRead, err := conn.Read(bufIncoming)
  if err != nil {
   LogEvent(strFromListenerNumber, err.Error())
   return
  }
  strMessage = string(bufIncoming[0:bytesRead-1])
  LogEvent(strFromListenerNumber + ": Remote IP " + strRemoteAddrSansPort, strMessage)
 } 
}
func LogEvent(strFrom string, strMessage string) {
 fmt.Println("From port " + strFrom + ": " + strMessage)
}

And here I'll end part 3!

Creating a Simple Golang Network Application, Part 2

Next I turned my thoughts to logging. If the processes are going to "log" things (eventually to a text file) I should probably create something to stub out that function.

Is that even the right word, stub? I used to think it was a placeholder so other parts of the application had something to call that could be fleshed out later. Apparently looking into the term I ended up discovering that it has a more specific meaning...and I'm not sure what it is called to create a function that does something, but the intention is to come back later to add or enhance the functionality; maybe the definition still fits.

But for my purposes here that doesn't matter. The point is, I wanted to have a separate, modular function call to do the work of printing stuff to the screen or dumping stuff to a text file.

So I created this:

func LogEvent(strFrom string, strMessage string) {

 fmt.Println("From " + strFrom + ": " + strMessage)
 
}

The function is called LogEvent(), and takes two string arguments. The strFrom is so the caller can insert the process (remember, asynchronous goroutines...) and strMessage is the message. This simplified form is going pretty much retain the previous incarnation's output, but under the hood I can greatly reduce duplicated code.

Also notice that I started using a naming convention of strNameOfFunction. I usually preface my variables with their type so I can keep track of them. The only time I didn't usually do that is using simple one-off single letter counters, like the "i" in the initial loop contained in main().

It started bugging me that my ListenToPort function didn't follow my variable naming convention. I had to rework it to use the logging function anyway, so I fixed it.

func ListenToPort(uint16Port uint16) {

 var strDiagMessage string

 strDiagMessage = "Listening to port " + strconv.FormatInt(int64(uint16Port),10)
 LogEvent(strconv.FormatInt(int64(uint16Port),10), strDiagMessage)
 
}

I created a string variable because, as I alluded previously, integers aren't strings. This was an easy way to make it clear that I'm converting something to a string and storing it in a nice little package to send off to another function. strDiagMessage is declared, then it's set to the diagnostic message for my testing and shuffled into LogEvent. I could have created another variable to store the name of the "port" it's listening on, but...meh.

The next item I started thinking about was communicating from the goroutines back to the calling process. Because the goroutines are asynchronous, doing anything that requires a check of the status of an individual goroutine requires some kind of communication channel to exchange data and "sync up" information. At this point I wanted to validate that the goroutines were running properly and could give a nod back to main() that those processes were ready.

This approach also meant I could get rid of the clunky timer. The timer is just a guesstimate of how long it will take all the goroutines to run and is rather sloppy. Time to modify some source code...

First, I won't need timers, so I remove the 'import "time"' line along with the time.Sleep and the associated "done" line as well as the comments referring to those lines. I use comments to explain what I'm doing for later reference mainly because I have the memory of a sieve, but I'm documenting everything here so for a short program like this I figure it's not really needed (and highly redundant when I'm elaborating in exposition.)

Next I create a channel. It's a Go method of intra-communication among asynchronous code; I'll spare you the details beyond the idea that if you want to pass data among routines, it's a handy data chute to slide information. And to be completely honest, channels seem a little tricky to get the hang of...

In my block of declarations, I create a channel called chanCommLine with:

chanCommLine := make(chan string)

Next I want to collect data from my goroutines. I change the definition of ListenToPort so it takes an additional argument:

func ListenToPort(uint16Port uint16, chanOutGoing chan<- string) {

Then I add a line to send both a "done" along with the "port number" (at this point, port number is a name-only thing...it's the port the particular routine is supposed to handle, not what it's actually doing. Don't get confused. Practically speaking it's an identifier of which goroutine is passing the message.)

 chanOutGoing <-"done " + strconv.FormatInt(int64(uint16Port),10)

The message sent into the channel is getting passed back to the main() function where something is listening for input from the channel conveyor belt. Of course something has to pass that channel to ListenToPort, so in main() where it's invoked I change the call to:

  go ListenToPort(i, chanCommLine)

Now I have to do something with that information. I add the following after all the go routines are spawned:

 i = 0

 for {

  strChanData:=<-chanCommLine
  if strChanData != "" {
   i++
   fmt.Println("Input detected from channel, loop " + strconv.FormatInt(int64(i),10) + " " + strChanData)
  }

  if i == 1024 {
   fmt.Println("Channels accounted for")
   break
  }
 }

What's happening here?

First I took the iterator from the previous loop and reused it, setting it to 0.

Next I create an infinite loop with "for" and inside create a string variable strChanData that is set to the data coming out of chanCommLine. Then, if the value isn't empty, another loop runs incrementing i and outputting a line telling the user that information was found from loop number something and at the end printing the data that came out of the channel.

When i hits 1024, the program prints that all the channels are accounted for and then "break" exits the infinite loop.

Notice this is comparing for 1024 while the loop spawning the goroutines counts to 1025? That's because of the point at which the comparison is being made. The loop creating the goroutines starts at 1, and once it's compared at the beginning of the final loop it will equal 1025 and not run again, leaving 1024 iterations. In this loop it starts at 0, runs a series of steps, then compares the state of the variable; if it's 1024 at the end of the last run, it stops. Rearranging the logic a bit could make it all consistent with one value (1024 or 1025) but I wanted the port counting to start at 1 and really, this isn't all that strange to figure out once you see the logic. I would wager that loop comparisons can be a huge source of difficulties and logic errors for new programmers, though.

How did I verify I have it running with the proper loop counts? I did a quick build ("go build" from the src directory) and ran my application with a "|grep 102" to see what popped out:

Input detected from channel, loop 102 done 874
From 1020: Listening to port 1020
From 1021: Listening to port 1021
From 1022: Listening to port 1022
From 1023: Listening to port 1023
From 1024: Listening to port 1024
Input detected from channel, loop 248 done 1020
Input detected from channel, loop 249 done 1021
Input detected from channel, loop 250 done 1022
Input detected from channel, loop 251 done 1023
Input detected from channel, loop 252 done 1024
From 102: Listening to port 102
Input detected from channel, loop 354 done 102
Input detected from channel, loop 1020 done 769
Input detected from channel, loop 1021 done 770
Input detected from channel, loop 1022 done 771
Input detected from channel, loop 1023 done 772
Input detected from channel, loop 1024 done 902

You can see the highest count is 1024. If the goroutines hit 1025, the lines starting with "From" would list a 1025, and if the channel loop hit 1025 then a line starting with "Input" would have a loop 1025 listed.

One important thing to note is that you can't treat the channel as if it were a variable. When something reads from the channel it is "emptied." In one version of the code I had tried to use the channel in a conditional check then output the contents in two separate statements; the number of loop iterations dropped to 512, only half of what was supposed to run. Why?

It happened because I was reading once as a conditional check, and in a second statement tried dumping the contents of the channel. What actually happened was it emptied when I checked the channel to evaluate for a condition then when the channel was used to print the contents it was getting the next item put into the channel, not the item used to check the condition.

On the bright side at least it was kind of obvious when the symptom was cutting the loop iterations in half...

A minor fix and voila! Communication from the goroutines to the caller! If you recall the earliest version of the program had a 2-second delay built in. This method of having channels tell main() that the goprocesses are all ready and listening means there should be a drastic drop in "do nothing" time. How long does the current version take to run?

I ran port_listen (the name of the application) on my somewhat old Mac running at 2.4 GHz on a Core 2 Duo processor using the "time" utility:

real 0m0.152s
user 0m0.010s
sys 0m0.010s

I'd say that's a bit of an improvement!

Here's what the source looks like at this point:

package main

import "fmt"
import "strconv"

func main() {

 var i uint16
 chanCommLine := make(chan string)
 
 for i = 1; i < 1025; i++ {
  go ListenToPort(i, chanCommLine)
 }

 i = 0

 for {
  strChanData:=<-chanCommLine
  if strChanData != "" {
   i++
   fmt.Println("Input detected from channel, loop " + strconv.FormatInt(int64(i),10) + " " + strChanData)
  }
  if i == 1024 {
   fmt.Println("Channels accounted for")
   break
  }
 }
}

func ListenToPort(uint16Port uint16, chanOutGoing chan<- string) {
 var strDiagMessage string

 strDiagMessage = "Listening to port " + strconv.FormatInt(int64(uint16Port),10)
 LogEvent(strconv.FormatInt(int64(uint16Port),10), strDiagMessage)
 chanOutGoing <-"done " + strconv.FormatInt(int64(uint16Port),10)
}

func LogEvent(strFrom string, strMessage string) {
 fmt.Println("From " + strFrom + ": " + strMessage)
}

This concludes part two!

Creating a Simple Golang Network Application, Part 1

Many moons ago I worked in a place that had an "incident" involving a network worm. It would hop from system to system, leaving behind itself as a payload before trying to find another machine to try logging into and repeating the spreading process.

I was reminded of this incident while troubleshooting a network issue for someone. What if something were on this user's network, poking around for access? How would she know?

What if I created a small application she could run on her Mac that would act as a kind of dumb honeypot; accepting connections and logging what the remote machine sent?

That's what set me on the path of using Go to create a simple program to do just that. I was vaguely familiar with the language and the resulting Go executable is statically linked; I can just send her the executable and it'll run, no need to make sure a set of libraries or framework was up to date before the program would run properly. And Go is also pretty fast, far faster than Python or Ruby.

I decided to chronicle the process of creating this application; perhaps it would be useful to a novice programmer wondering if anyone else created small programs with a similar process. I know there are times I would have found that kind of information useful. So...I'm leaving this here.

The first step was figuring out what I wanted the program to do. This was going to be a simple, small application. I wanted it to be easy enough that a kind of non-technical person could easily be talked through the steps of running the application. I wanted the program to listen to all the service ports, the ones commonly probed for vulnerabilities, and accept TCP connections and then log whatever was sent to those ports (passwords or commands, for example) to a text file.

I started off by making a simple "skeleton" of the application, modeling certain behaviors I could use as the bones of the program.

package main

import "fmt"
import "strconv"
import "time"

func main() {

 var i uint16
 
 for i = 1; i < 1025; i++ {

  go ListenToPort(i)
  
 }

 // This sleep and print is just a debugging/testing thing. The for loop using 
 // "go ListenToPort" is asynchronous...so we have to wait or it just finishes.
 time.Sleep(2 * time.Second)
 fmt.Println("Done!\n")
 
}

func ListenToPort(port uint16) {

 fmt.Println("Listening to port " + strconv.FormatInt(int64(port), 10))
 
}

Package main -> this tells the compiler this is the "main" application file with the main() function, instead of a package that is treated like a library to be compiled in the workspace pkg directory.

Import -> fmt, strconv and time are built-in libraries to the language. Fmt handles string handling and formatting, strconv handles converting strings to and from other types, and time lets us play with sleep.

main -> This function consists of just a for loop to run from 1 to 1024 and increment by one each time. Each iteration of the loop spawns a goroutine call to ListenToPort with the current iterator value, which later will be used to listen to the network ports. For now it just models the calling behavior validating that my looping logic will work in the earliest stages.

The "go" keyword means "this is to be run as a goroutine without returning a value or even waiting for any feedback." The application will spit out 1,024 asynchronous routines like cards shooting from the hands of a magician playing 1,024 card pickup before finishing the loop. The timing of these routines, separately scheduled, will finish at unpredictable moments later and may not even finish in order.

time.Sleep(2 * time.Second) and the fmt.Println-> Remember how I said the "go" keyword spawns go processes without regard for returning values or waiting for things to complete? Yeah, without this "let's wait" thing the system will almost immediately spit out "Done!" This means the application will finish before the goroutines have a chance to do anything.

This sleeping pause just gives the application some time to show some output. This is a quick-and-dirty thing...don't judge me! I'm simply forcing main() to give some breathing room to validate my loop logic.

The Main() function is closed up, and I then define a placeholder for a useful ListenToPort function with ListenToPort(port uint16); func means this is a function, ListenToPort is the name of that function, and "port" is the name of the variable I want to work with and uint16 means unsigned 16 bit integer since that should be enough to hold the number of ports I could want to iterate through.

The Println call is kind of complicated for the purposes of a simple test taking shape, but basically it is saying "Print 'Listening to port" and append the port number passed to me." Because it's an integer, I have to convert the uint16 into a string or a type error will be spat out by the compiler.

There doesn't seem to be a straightforward way of changing the variable into a string but FormatInt will do the conversion if the variable is a 64 bit int; so I cast port to a 64 bit integer using int64. The 10 is because FormatInt takes a "base number" argument and I'm using base 10 counting.

This source code compiles and spits out numbers...almost in order...to the command prompt, and after three seconds pass it says, "Done!"

Pretty simple, eh?

I'll end part one here...

Does Anyone Use OS X Guest User?

Our company has a few shared spaces...conference rooms...in which some Mac Minis are set up for general purpose use. You know, Vidyo video conferencing, presentations, web demos...nothing too demanding.

We've had to balance the needs of our users with our need for management and security; ease of use for our users, while mitigating potential problems that come from a shared system.

Lets try AD!

At first we had the Macs allowing people to log in via Active Directory. They had their own logins under which they could save files.

The problem was they would log in and forget to log out, leaving not just (potentially sensitive) files available but often access to their online accounts like Google. They also had the default Dock configuration, so sometimes users would try reinstalling applications that already existed on the Mac (like Chrome.)

Another issue we frequently ran into involved Keychain, the built-in OS X password management system. Users would change their password from a Windows workstation, and the next time they used the shared Mac they would often fail to update the Keychain password, leading to a constant stream of Keychain access prompts. Confused users would quickly go from annoyed to extremely frustrated, especially when trying to give a presentation or in the middle of video conferences.

The bit rot of accumulating login accounts and eating away at storage space with each new login instance was a relatively minor inconvenience compared to the Keychain password mismatches, from the usability point of view.

Let's try Guest!

The next obvious solution is to try Guest. A feature built into OS X, Guest allows users to log in without a password for a temporary session; upon logging off, all the files are deleted. Coupled with the "inactivity logout" setting found by clicking the "Advanced..." button in the Security & Privacy pane, this seemed like the ideal setup!

But alas, it wasn't.

Three big issues for the end users could have been solved with what should have been some simple tweaks to default settings. When using a system that allows temporary "guest" accounts, most Unix systems use a skeleton or template directory from which to copy files. That didn't seem to work in this case.

Our users wanted to use Chrome as a default browser; with each login Safari was reinitialized as the default. Second, the users wanted the Chrome icon available in the dock as a default available application to choose from; the idea of opening Applications or Spotlight first confused the hell out of them. And third, every time the users wanted to use Office, it would have them go through the "first run" wizard. That was not confusing, but it was understandably annoying.

But the biggest issue was one we never could figure out why it was happening. Keychain. Again.

If the system was left on and had a few login/logoffs of Guest, suddenly Keychain would complain about needing a password. Guest works, in part, by generating a random password and transparently using it where needed for the Guest login session. These temporary keychains should have disappeared when Guest logged off since the Library files, along with the rest of the home directory, were deleted when Guest logged off.

A quick sanity check from a local administrator account via SSH showed that yes, that Guest folder was actually gone when the user logged off.

But Keychain acted like the old file was still held open or cached. File's gone. Keychain's not running. LSOF didn't show an open file holding anything under a guest home directory open. Yet the only thing that cleared the errors was a complete system reboot.

A minor irritation was the number of times I'd check on the Mac only to find the Guest login still logged in after several hours, prompting a nonexistent person in the room if they were really sure they wanted to delete all the files and log off. Are you sure? Are you really sure?

Great...users would still constantly forget to log out, but instead of doing what I told it to do and log out when the inactivity timer was triggered, it would get hung up on the logoff prompt. The only thing it really did right was when it was forced to log off, it deleted all the Guest home directory files so the user's private files and potential data leaks were removed.

Guest, in theory, is a way to create a kind of temporary session for your Mac. I thought there had to be a way to customize some of the configuration (through altering some preference files in a template directory) for little things like , but some research on the Interwebz told me that there may have been a way to do that with older version of OS X, but it was rendered obsolete in newer versions.

I contacted our account rep at Apple and asked to get in touch with an engineer, which they obliged. I sent them a description of what was happening and what we were trying to do with regards to registering Office in the temporary session, customizing the default web browser selection and adding an icon to the dock, and most perplexing, the Keychain errors. I was hoping that an Apple engineer would know more about how to alter these settings (and get whatever was holding that keychain open to release it...), or set me straight if I were missing a particular way to manage the Mac clients that could control the interface.

The response:

The guest account doesn’t quite fit your requirements. You could create your own guest account and then manipulate the prefs files for default browser, office licensing file, etc. 

Other than that there are products like Deep Freeze or FileWave to manage lab or kiosk type machines. 

For the keychain issue try this on a test machine first

Delete the "User Template/English.lproj/Library/Keychains/" folder.

Guest must be suitable only for a very narrow set of use cases. I verified there wasn't a Keychains folder in that template directory, too.

It seems Guest is really kind of a worthless feature in most cases, as far as I can tell.

Last try...Deep Freeze

That brings me to the final round. We purchased several Deep Freeze licenses and created a new user named Login. I configured Login to have no password and tweaked the default home to have minimal dock icons, add the icons that were useful, told the Chrome browser it should be the default, and registered Office with generic user settings.

The login is local, so there's no worry about a password change causing Keychain to get out of sync and start spitting errors.

I had limited exposure to Deep Freeze on the Windows platform before, and no exposure to a Mac version of the application. Basically the application redirects filesystem writes to a temporary location that is wiped at restart. Unlike Guest, that only deleted files from the Guest home directory, Deep Freeze will clear changes made to the whole system. One of my favorite cathartic activities when I ran Deep Freeze on Windows was to delete the Windows subdirectory, or delete other system subfolders until Windows started throwing errors before crashing. Upon reboot, all went back to the previous configuration, system files and all. 

Note you should only do that in a "frozen" state. If the machine were thawed and the files were deleted, you were kind of boned.

Monitoring the Freeze status came from an extended report to Apple Remote Desktop, unlike the Windows version that connected to a management control program. 

The user I named "Login" had no password, making it easy to use for our users.

Deep Freeze has a setting that causes the system to reboot when you log out, causing the system to reset. So far I haven't run into a system waiting at a logoff prompt as I did for Guest, so reboots with Deep Freeze seem to be working (and resetting) fine.

Office was registered before being frozen for the Login user, so users no longer get that prompt.

And of course the Dock has the icons for the most used applications, along with Chrome set to the default web browser.

My only complaint so far with Deep Freeze has been the inability to get along with Apple's Core Storage volume driver nor Fusion Drive. We didn't have Fusion Drive-equipped systems...but I thought I'd throw that in there.

I think the Core Storage system, which is kind of like a pluggable file system for the Mac that handles features like File Vault, doesn't get along with Deep Freeze because I suspect Deep Freeze inserts itself at a similar driver level.

I was able to install Deep Freeze on most of our existing systems because I think we accidentally broke the default-Core-Storage-enabled volumes when we upgraded the hard drives in the Mac Mini's. If you didn't upgrade them and Core Storage is enabled, the fix is basically going to involve a reinstall of the operating system. That...is annoying. Especially after spending time tweaking all the settings and getting everything set up until you get to the Deep Freeze install stage, just to have it spit an error back at you.

In summary...

Does anyone actually use Guest on their Mac? If you do, what is it used for? It seems unsuitable for use as a kiosk. It can't be tweaked with regard to preferences or icons in the dock. I contacted an Apple engineer hoping I was stupidly overlooking something simple, only to be told that I was asking too much of the Guest capabilities (was I really asking all that much?)

In my opinion, with Guest being so limited, Apple should just eliminate it. Especially with that lingering Keychain bug...

Goodbye Mail.app, Hello Thunderbird

Oh, Mail.app...our love/hate relationship blossomed so quickly. I was willing to overlook your quirks for so long but some recent interactions pushed the limit beyond my tolerance levels.

I overlooked your occasional decision to just stop receiving mail. Lunchtime would roll around and I realized that I didn't get the announcement that it was ready...I'd check Gmail's web interface and the message was there, sometimes with other messages in the queue as well, but your interface was blank to new messages. I'd close you out and re-launch and the new messages would pour in. But we all have occasional lapses in attention. As long as my phone or Pavlovian response would remind me to check when you were acting up, I guess that's good enough to forgive that occasional lapse.

Then there were the times you didn't let me eject a disk. Well, an SD card, but it's treated as a disk. I'd insert the chip, and then I'd attach an image stored on the card...it was the simplest way to get images from the camera to you...to a coworker so he could retouch it for use in our badge printer. Then I'd try to eject the disk after sending the email and the operating system insisted the disk was in use. Lsof would confirm that you were holding the file open. Why?

You'd never tell me. But if I exited and relaunched you, suddenly the SD card could be ejected. I figured it was just your way of pulling a silly prank on me. Maybe it was your way of protesting frivolous use of fclose(). Or you were trying to train me to copy files to a local directory before using them as attachments. Perhaps fclose() is just hard? I don't know. You just persisted in teasing me, never telling me why.

But some sins just couldn't be forgiven. I mean, the whole "forgetting to get mail" was close...especially when I'm expected to reply to requests for help. I was fortunate enough to have multiple notification systems for that, so,...haha! Joke was on you!

Crashing the operating system was over the line. I'm not sure how it happened, exactly. I suspected it was triggered by something in the formatting of a quote from one of our vendors; I'd view the message, and part of the quote...listing all the options in the system configuration...was missing. Within a minute the operating system belched a warning that all the application memory was exhausted. I couldn't stop it once the spiral started. The interface became unresponsive and soon locked up. Secure shell no longer answered attempts to connect. The operating system just giggled at me in a permanently frozen grin. I had to power cycle the machine. If you relaunched and tried previewing that message again, BAM, memory exhaustion followed with another hard reset. This was a quirk I couldn't work around.

Well, technically I could use Gmail's web interface to clear the message, which is what I did. But if I get another message from that vendor with a quote attached, I knew it would lead to another round of hard restarts and filesystem checks. I was stuck waiting for what amounted to a denial of service attack straight to my inbox.

So it was at this point I had no choice but to say goodbye. It turns out it seems there aren't a lot of free mail clients for OS X. Maybe I didn't look hard enough. But that's okay. I decided to give Thunderbird a try. It's not perfect; it's a cross-platform client, and it shows in the way it handles the interface. It doesn't quite feel like it does things in the "Macintosh Way." But it's decent. So far it doesn't crash on me when viewing attachments. I haven't exhaustively checked everything, but I'm sure I'll discover handling issues over time, if they're there.

And you didn't make it easy to leave you, Mail! I had to search to figure out how to set Thunderbird as my "default" mail client. Turns out I had to launch YOU again and change the setting from your menus! It made no sense to me!

There are times that I miss you. I think of you every time I search for an email. It's impossible! Well, as long as I use Thunderbird's search. It's painful. There's a setting in the preferences to allow Spotlight to index messages; I end up using that instead. It's irritating, having to use a service outside the application to search for something stored in the application. But at least it works.

Well, mostly. Even after making the "default mail client change," opening messages from Spotlight search results would open using you! There's no easy way to find a way to change that. I ended up opening a question on the Apple Stack Exchange site to try fixing it.

And of course Thunderbird acts a little weird about attachments. It has a neat feature where if I type the word "attached"...like, "I attached a photo to edit for Bob's badge!"...Thunderbird will ask me if I want to attach a file. And I can navigate to the folder and attach the file in question. It appears as a "file" list in a pane near the top of my mail composer.

But I can also drag and drop the picture into the composition window. Then the image appears right in the email, much like when I attach files into your messages, Mail!

But...it's not consistent. I sent the same image in both ways, and when I received them, it was like the message was embedding the file in two different ways. How weird and annoying. I also realized that attachments don't show up in the reader. PDF's look like attached icons, and to view them I have to open them. I can't drag and drop them to other applications; with Mail, I can have our online KanBan board opened to a card where I can drop attachments to upload to that system. So I'd get a quote from a vendor, and on the card where I was tracking the purchase, I would just drag the PDF from the email to the KanBan card and it would upload. With Thunderbird I had to save the file to the disk first. Another annoyance.

There are just too many seemingly common file formats that Thunderbird doesn't know how to display inline. An annoyance. About on par with the times you wouldn't let me eject a disk after sending a file. Tolerable, I guess.

Now I'm waiting for a showstopper to appear in Thunderbird. It took awhile before I gave up on you, Mail.app. It took a long time before I hit something that I just couldn't make excuses for anymore. And I'm still sad to say goodbye; you have so much potential! I wanted you to work well. But sometimes...it felt like you just didn't get enough love from your programming team. I saw people complaining about issues similar to my memory exhaustion problem back in October, nearly 6 months ago! Couldn't this have been addressed in a patch by now?

Now I'm getting into the groove of using Thunderbird. Even if you did fix these issues, I don't know if I'll want to come back. Or I won't want to come back without a really good incentive. I suppose it's possible. Maybe someday I'll forget about your irritations and wonder why I ever left you. I'll reinstall the operating system and think it's best to just use the default client rather than go through the trouble of reconfiguring Thunderbird all over again...there sure a lot of settings to tweak, after all. Maybe at that point I'll sigh and dread making sure all my settings are properly set and decide it's best to just return to you, after writing this post becomes a distant memory.

Or the Thunderbird team will have a simple way to export settings.

Mac Thunderbolt Display Serial Numbers from the Command Line

A display recently "died" on one of our users. It no longer lit...turning it into a shiny work of pitch-black art...but the hub functions still worked, and remotely connecting to the system with Apple Remote Desktop shows two displays (the Thunderbolt and connected laptop.) 

In order to check the status of the Applecare warranty, I needed the serial number. This system had been previously removed from its stand and connected to a mounting arm, so I couldn't guarantee the hardware matched what was printed on the bottom of the stand. Uh-oh.

Fear not! The Mac actually has a number of fairly awesome tools available if you know where to look. The system I had connected to it was running 10.9 and the system information application wouldn't tell me the serial number of the display. 

The workaround is to open the terminal, then run:

sudo system_profiler SPDisplaysDataType

This spits out some helpful information, and buried in that information dump is a section called Displays. In that is "Thunderbolt Display:", and under that is the serial number!

Seriously handy for system information, especially if you have to ssh into a user system and look up hardware information. For more information take a gander at:

system_profiler -usage

Upgrading GoLang From Source (Using Git)

I used to have a pretty straightforward way of upgrading my GoLang installation on the Raspberry Pi (ARM Linux). A few quick commands and a long wait as it recompiled and everything was right as rain. Then the Go team decided everything would go to Git and I had to follow suit or keep fighting to get Mercurial to work with a neglected repo.

So how do you upgrade using Git?

From the home directory where I keep my Go subdirectory:

cd go/src
git fetch
git checkout <tag>
./all.bash

Wait until baked to a golden brown and remove from the oven. Voila'.

How do you know what tag to use? And when to upgrade?

From the golang-announce list, of course. You can manually check there for latest release information or subscribe to get the announcements to your inbox.