Go has a mechanism in the "sync" package called a mutex, used to lock something against alterations when concurrent routines might try altering data. From what I could find it's often recommended that when possible you should use alternate methods like channels and waitgroups to guarantee concurrency doesn't upset your variables/structs/etc, but mutexes should be used for exclusive access when working on caches and variable state.
But how do they work? As in, how should I model their behavior for implementation in my application?
Let's try some simple applications to see how mutexes work.
What I want to do is have a struct whose values are "protected" and altered by only one section of code at a time. Can a mutex do that?
Test One
First test I'll spin off some goroutines to hammer a change on a struct, with another routine locking the struct.
First test I'll spin off some goroutines to hammer a change on a struct, with another routine locking the struct.
// How do mutexes work on a struct? Let's see if this works the way I hope it does... package main import ( "fmt" "sync" "time" ) func main() { type ProtectThese struct { secure sync.Mutex firstname string lastname string } // Create an instance of the structs var protected ProtectThese // And a cheap flag for quitting the main() goroutine var Stop bool = false // What time is it? StartTime := time.Now() // Spin off a goroutine that just locks protected and waits go func() { // Lock it protected.secure.Lock() // Tell us when it was locked fmt.Println("Locked at " + time.Since(StartTime).String()) // Create a timer WaitForIt := time.NewTimer(time.Minute * 1) // After the timer elapses <-WaitForIt.C // Unlock the struct protected.secure.Unlock() // All done return }() // Another goroutine will try writing a value to firstname go func() { // An infinite loop because...why not? for { protected.firstname = "John" } }() // This goroutine will write a value to lastname go func() { // See the previous routine for { protected.lastname = "Doe" } }() // And yet another goroutine does nothing but reads values from protected to the stdout go func() { // A couple quick flags var firstchanged bool = false var lastchanged bool = false // Create a ticker to check the status of the struct periodically ticker := time.NewTicker(time.Nanosecond * 5) for range ticker.C { if protected.firstname == "John" && firstchanged == false { // Did the first name get filled in? fmt.Println("Firstname changed at " + time.Since(StartTime).String()) firstchanged = true } if protected.lastname == "Doe" && lastchanged == false { // Did the last name change? fmt.Println("Lastname changed at " + time.Since(StartTime).String()) lastchanged = true } if firstchanged == true && lastchanged == true { // Both are done. Change the running flag. Stop = true } } }() // This basically spins wheels until the variable for stopping is set // Checks every 50 milliseconds to see if we should quit for Stop == false { Waiting := time.NewTimer(time.Millisecond * 50) <-Waiting.C } }
What's happening here?
Main() has only a few jobs to perform. It creates a struct holding 2 strings and a mutex, it creates an instance of that struct, it gets the current time, and creates a little flag variable (old habits die hard) and then checks every 50 milliseconds for that variable to become true.
There are also 4 goroutines:
Goroutine(a) locks the mutex in the struct, prints out the time since Main took the current time that the lock was set, then sits for a minute before unlocking it.
Goroutine(b) keeps trying to set the firstname string in the struct to John.
Goroutine(c) keeps trying to set the lastname string in the struct to Doe.
Goroutine(d) checks every 5 nanoseconds whether the firstname and lastname have changed, and if they did, prints the time since main() set the current time that it was changed. When both firstname and lastname are set, goroutine(d) also sets the "Stop" variable telling main() to quit.
Pretty simple. What happens when it's run?
Locked at 27.549µs
Firstname changed at 174.431µs
Lastname changed at 251.301µs
That's...fast. It ran on the order of microseconds. Pretty good indication that goroutine(a)'s lock didn't stop goroutine(b) and (c) from altering the struct.
Test Two
Let's try something a little different.
// How do mutexes work on a struct? Let's see if this works the way I hope it does... package main import ( "fmt" "sync" "time" ) func main() { type ProtectThese struct { secure sync.Mutex firstname string lastname string } // Create an instance of the structs var protected ProtectThese // And a cheap flag for quitting the main() goroutine var Stop bool = false // What time is it? StartTime := time.Now() // Spin off a goroutine that just locks protected and waits go func() { // Lock it protected.secure.Lock() // Tell us when it was locked fmt.Println("Locked at " + time.Since(StartTime).String()) // Create a timer WaitForIt := time.NewTimer(time.Minute * 1) // After the timer elapses <-WaitForIt.C // Unlock the struct protected.secure.Unlock() // All done return }() // Another goroutine will try writing a value to firstname go func() { // Let's wait a little bit before trying to set this. Pause := time.NewTimer(time.Second * 30) <-Pause.C // An infinite loop because...why not? for { protected.firstname = "John" } }() // This goroutine will write a value to lastname go func() { // See the previous routine for { protected.lastname = "Doe" } }() // And yet another goroutine does nothing but reads values from protected to the stdout go func() { // A couple quick flags var firstchanged bool = false var lastchanged bool = false // Create a ticker to check the status of the struct periodically ticker := time.NewTicker(time.Nanosecond * 5) for range ticker.C { if protected.firstname == "John" && firstchanged == false { // Did the first name get filled in? fmt.Println("Firstname changed at " + time.Since(StartTime).String()) firstchanged = true } if protected.lastname == "Doe" && lastchanged == false { // Did the last name change? fmt.Println("Lastname changed at " + time.Since(StartTime).String()) lastchanged = true } if firstchanged == true && lastchanged == true { // Both are done. Change the running flag. Stop = true } } }() // This basically spins wheels until the variable for stopping is set // Checks every 50 milliseconds to see if we should quit for Stop == false { Waiting := time.NewTimer(time.Millisecond * 50) <-Waiting.C } }
What's happening here?
Not much has changed; I just altered goroutine(b) to have a 30 second pause. What happens when I run it?
Locked at 44.836µs
Lastname changed at 160.966µs
Firstname changed at 30.000173283s
Shows that the timers are working. Still shows the mutex isn't doing anything.
Test Three
Time to try wrapping one of the goroutines in a mutex lock.
// How do mutexes work on a struct? Let's see if this works the way I hope it does... package main import ( "fmt" "sync" "time" ) func main() { type ProtectThese struct { secure sync.Mutex firstname string lastname string } // Create an instance of the structs var protected ProtectThese // And a cheap flag for quitting the main() goroutine var Stop bool = false // What time is it? StartTime := time.Now() // Spin off a goroutine that just locks protected and waits go func() { // Lock it protected.secure.Lock() // Tell us when it was locked fmt.Println("Locked at " + time.Since(StartTime).String()) // Create a timer WaitForIt := time.NewTimer(time.Minute * 1) // After the timer elapses <-WaitForIt.C // Unlock the struct protected.secure.Unlock() // All done return }() // Another goroutine will try writing a value to firstname go func() { // Set the lock here in addition to the previous goroutine protected.secure.Lock() // An infinite loop because...why not? for { protected.firstname = "John" } // Unlock protected.secure.Unlock() }() // This goroutine will write a value to lastname go func() { // See the previous routine for { protected.lastname = "Doe" } }() // And yet another goroutine does nothing but reads values from protected to the stdout go func() { // A couple quick flags var firstchanged bool = false var lastchanged bool = false // Create a ticker to check the status of the struct periodically ticker := time.NewTicker(time.Nanosecond * 5) for range ticker.C { if protected.firstname == "John" && firstchanged == false { // Did the first name get filled in? fmt.Println("Firstname changed at " + time.Since(StartTime).String()) firstchanged = true } if protected.lastname == "Doe" && lastchanged == false { // Did the last name change? fmt.Println("Lastname changed at " + time.Since(StartTime).String()) lastchanged = true } if firstchanged == true && lastchanged == true { // Both are done. Change the running flag. Stop = true } } }() // This basically spins wheels until the variable for stopping is set // Checks every 50 milliseconds to see if we should quit for Stop == false { Waiting := time.NewTimer(time.Millisecond * 50) <-Waiting.C } }
What's happening here?
This time in addition to goroutine(a) setting the mutex, I added the mutex around goroutine(b) as well. This is not entirely safe in that there is a race condition; it appears that goroutine(a) is spun up faster than (b), so it sets the lock sooner in every test I've run. If something were to delay (a) running, (b) could get to it first. Just something to remember.
What happens when this one is run?
Locked at 23.15µs
Lastname changed at 114.786µs
Firstname changed at 1m0.000142691s
Ah-ha! Goroutine(b) is stuck until the lock set by (a) is unlocked! This confirms that the mutex isn't magically wrapping the struct and set by whatever calls it...it's more like a flag, and any code that might interfere with it must still be mindfully set to check that flag before altering things. In other words, if you want the struct to be protected, you still have to place checks in your code where the code might stomp on the state of the struct; the mutex doesn't lock the struct from random changes.
Some of the sources I found online could be interpreted as saying that the mutex only locks the data right under it (that seems strange?) in the declarations, which here would mean firstname is protected, but lastname isn't. Let's test that.
Test Four
// How do mutexes work on a struct? Let's see if this works the way I hope it does... package main import ( "fmt" "sync" "time" ) func main() { type ProtectThese struct { secure sync.Mutex firstname string lastname string } // Create an instance of the structs var protected ProtectThese // And a cheap flag for quitting the main() goroutine var Stop bool = false // What time is it? StartTime := time.Now() // Spin off a goroutine that just locks protected and waits go func() { // Lock it protected.secure.Lock() // Tell us when it was locked fmt.Println("Locked at " + time.Since(StartTime).String()) // Create a timer WaitForIt := time.NewTimer(time.Minute * 1) // After the timer elapses <-WaitForIt.C // Unlock the struct protected.secure.Unlock() // All done return }() // Another goroutine will try writing a value to firstname go func() { // Set the lock here in addition to the previous goroutine protected.secure.Lock() // An infinite loop because...why not? for { protected.firstname = "John" } // Unlock protected.secure.Unlock() }() // This goroutine will write a value to lastname go func() { // Set the lock protected.secure.Lock() // See the previous routine for { protected.lastname = "Doe" } // Unlock protected.secure.Unlock() }() // And yet another goroutine does nothing but reads values from protected to the stdout go func() { // A couple quick flags var firstchanged bool = false var lastchanged bool = false // Create a ticker to check the status of the struct periodically ticker := time.NewTicker(time.Nanosecond * 5) for range ticker.C { if protected.firstname == "John" && firstchanged == false { // Did the first name get filled in? fmt.Println("Firstname changed at " + time.Since(StartTime).String()) firstchanged = true } if protected.lastname == "Doe" && lastchanged == false { // Did the last name change? fmt.Println("Lastname changed at " + time.Since(StartTime).String()) lastchanged = true } if firstchanged == true && lastchanged == true { // Both are done. Change the running flag. Stop = true } } }() // This basically spins wheels until the variable for stopping is set // Checks every 50 milliseconds to see if we should quit for Stop == false { Waiting := time.NewTimer(time.Millisecond * 50) <-Waiting.C } }
What's happening here?
Both goroutines(b) and (c) are wrapped in the call to the mutex lock. Output from the application:
Locked at 24.157µs
Firstname changed at 1m0.000158927s
^C
Ooh. I had to control-C that. What happened?
The key probably lay in here:
// Spin off a goroutine that just locks protected and waits go func() { // Lock it protected.secure.Lock() // Tell us when it was locked fmt.Println("Locked at " + time.Since(StartTime).String()) // Create a timer WaitForIt := time.NewTimer(time.Minute * 1) // After the timer elapses <-WaitForIt.C // Unlock the struct protected.secure.Unlock() // All done return }() // Another goroutine will try writing a value to firstname go func() { // Set the lock here in addition to the previous goroutine protected.secure.Lock() // An infinite loop because...why not? for { protected.firstname = "John" } // Unlock protected.secure.Unlock() }()
Goroutine(a) locks the struct, waits a minute, then unlocks and quits. The goroutine is gone.
Goroutine(b) then gets the struct next; it locked the struct, then enters an infinite loop setting the firstname repeatedly. The unlock() is never called!
Let's try fixing that.
Test Five
// How do mutexes work on a struct? Let's see if this works the way I hope it does... package main import ( "fmt" "sync" "time" ) func main() { type ProtectThese struct { secure sync.Mutex firstname string lastname string } // Create an instance of the structs var protected ProtectThese // And a cheap flag for quitting the main() goroutine var Stop bool = false // What time is it? StartTime := time.Now() // Spin off a goroutine that just locks protected and waits go func() { // Lock it protected.secure.Lock() // Tell us when it was locked fmt.Println("Locked at " + time.Since(StartTime).String()) // Create a timer WaitForIt := time.NewTimer(time.Minute * 1) // After the timer elapses <-WaitForIt.C // Unlock the struct protected.secure.Unlock() // All done return }() // Another goroutine will try writing a value to firstname go func() { // Set the lock here in addition to the previous goroutine protected.secure.Lock() // Loop to set the firstname for protected.firstname != "John" { protected.firstname = "John" } // Unlock protected.secure.Unlock() }() // This goroutine will write a value to lastname go func() { // Set the lock protected.secure.Lock() // See the previous routine for protected.lastname != "Doe" { protected.lastname = "Doe" } // Unlock protected.secure.Unlock() }() // And yet another goroutine does nothing but reads values from protected to the stdout go func() { // A couple quick flags var firstchanged bool = false var lastchanged bool = false // Create a ticker to check the status of the struct periodically ticker := time.NewTicker(time.Nanosecond * 5) for range ticker.C { if protected.firstname == "John" && firstchanged == false { // Did the first name get filled in? fmt.Println("Firstname changed at " + time.Since(StartTime).String()) firstchanged = true } if protected.lastname == "Doe" && lastchanged == false { // Did the last name change? fmt.Println("Lastname changed at " + time.Since(StartTime).String()) lastchanged = true } if firstchanged == true && lastchanged == true { // Both are done. Change the running flag. Stop = true } } }() // This basically spins wheels until the variable for stopping is set // Checks every 50 milliseconds to see if we should quit for Stop == false { Waiting := time.NewTimer(time.Millisecond * 50) <-Waiting.C } }
What's happening here?
Just added a quick check in the goroutines that set firstname and lastname so once the variable has the value set, they unlock and quit. Running the application yields the following output:
Locked at 30.974µs
Firstname changed at 1m0.000698523s
Lastname changed at 1m0.000794631s
That seems to show that yup, it locks for both those values in the struct.
So at this point it seems that you can lock the whole struct by just have a sync.Mutex included, and it's safe from alterations from code that you specifically wrap in a call to lock and unlock; if you forget, though, code that isn't wrapped in a fuzzy lock() blankie will merrily alter the values of your struct without batting an eye.
One last thing...is code trying to do something to the struct blocked, like channels get blocked waiting for a message to be pulled? Or do they try repeatedly? What does goroutine(b) do as it's currently written?
Test Six
// Another goroutine will try writing a value to firstname go func() { // Announce what I'm doing fmt.Println("About to lock struct so I can change firstname at " + time.Since(StartTime).String()) // Set the lock here in addition to the previous goroutine protected.secure.Lock() // I locked it! fmt.Println("Locked struct to change firstname at " + time.Since(StartTime).String()) // Loop to set the firstname for protected.firstname != "John" { protected.firstname = "John" } // About to unlock fmt.Println("About to unlock struct for firstname at " + time.Since(StartTime).String()) // Unlock protected.secure.Unlock() // And done... fmt.Println("Finished with altering firstname at " + time.Since(StartTime).String()) }()
What's happening here?
All I've changed is goroutine(b) so it has some additional announcements to STDOUT. Here's what the new output says:
Locked at 25.739µs
About to lock struct so I can change firstname at 121.747µs
Lastname changed at 1m0.000195611s
Locked struct to change firstname at 1m0.000158478s
Firstname changed at 1m0.000593511s
About to unlock struct for firstname at 1m0.000590839s
Finished with altering firstname at 1m0.000960257s
To me this shows that the mutex blocks access; the process just waits. But that could be because of the structure of the code; let's try constantly hammering it in a loop again.
Test Seven
// Another goroutine will try writing a value to firstname go func() { // Loop to set the firstname for protected.firstname != "John" { // Announce what I'm doing fmt.Println("About to lock struct so I can change firstname at " + time.Since(StartTime).String()) // Set the lock here in addition to the previous goroutine protected.secure.Lock() // I locked it! fmt.Println("Locked struct to change firstname at " + time.Since(StartTime).String()) // Make the change protected.firstname = "John" // About to unlock fmt.Println("About to unlock struct for firstname at " + time.Since(StartTime).String()) // Unlock protected.secure.Unlock() // And done... fmt.Println("Finished with altering firstname at " + time.Since(StartTime).String()) } }()
What's happening here?
Once again it's a relatively minor change in goroutine(b); now everything is encased in a loop instead of just the "let's make a change" bit. What is the output?
Locked at 36.342µs
About to lock struct so I can change firstname at 155.914µs
Lastname changed at 1m0.000200095s
Locked struct to change firstname at 1m0.000167036s
Firstname changed at 1m0.001612157s
About to unlock struct for firstname at 1m0.001605558s
Finished with altering firstname at 1m0.003361782s
Still blocking while the lock is set; it's not constantly retrying the loop or the "About to lock" message would keep repeating. I'm fairly confident that if you have a second process trying to do something on a struct you've locked from another process, it'll just block until the mutex is unlocked!
One last bit of fun, just to see what effect it would have; I wrapped the routine that checked for the status of the struct in a lock vs. no lock to see how many times it checked (after lowering the lock from goroutine(a) to 5 seconds);
// And yet another goroutine does nothing but reads values from protected to the stdout go func() { // A couple quick flags var firstchanged bool = false var lastchanged bool = false // Create a ticker to check the status of the struct periodically ticker := time.NewTicker(time.Nanosecond * 5) for range ticker.C { // Add one to the counter counter = counter + 1 // Lock it //protected.secure.Lock() if protected.firstname == "John" && firstchanged == false { // Did the first name get filled in? fmt.Println("Firstname changed at " + time.Since(StartTime).String()) firstchanged = true } if protected.lastname == "Doe" && lastchanged == false { // Did the last name change? fmt.Println("Lastname changed at " + time.Since(StartTime).String()) lastchanged = true } if firstchanged == true && lastchanged == true { // Both are done. Change the running flag. Stop = true } // Unlock it //protected.secure.Unlock() } }()
You can see the part I commented out, of course, to enable to lock. The results were kind of interesting (I assume it's because the check just blocked for the vast majority of 5 seconds...)
Without the lock: "I checked the status of the struct 124 times."
With the lock: "I checked the status of the struct 222896 times."
Maybe something to keep in mind if you have mutexes, a number of goroutines hitting excluded data, and performance issues!
No comments:
Post a Comment