When creating a "thing" that I didn't want other functions or operations to touch when something else might be doing something that could affect that thing, I would create a variable...usually a global-scoped variable...that I'd set with a particular value. Then I'd have code check that variable before doing anything that might upset other functions by altering the state of data. I usually called this a flag, but I'm sure in some cases it was more appropriate to call it a semaphore or...I'm not sure. It's probably against "proper" coding standards to do that. But it seemed to work as long as I was careful and commented what I was doing.
Go has a mechanism in the "sync" package called a mutex, used to lock something against alterations when concurrent routines might try altering data. From what I could find it's often recommended that when possible you should use alternate methods like channels and waitgroups to guarantee concurrency doesn't upset your variables/structs/etc, but mutexes should be used for exclusive access when working on caches and variable state.
But how do they work? As in, how should I model their behavior for implementation in my application?
Let's try some simple applications to see how mutexes work.
What I want to do is have a struct whose values are "protected" and altered by only one section of code at a time. Can a mutex do that?
Test One
First test I'll spin off some goroutines to hammer a change on a struct, with another routine locking the struct.
// How do mutexes work on a struct? Let's see if this works the way I hope it does...
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type ProtectThese struct {
secure sync.Mutex
firstname string
lastname string
}
// Create an instance of the structs
var protected ProtectThese
// And a cheap flag for quitting the main() goroutine
var Stop bool = false
// What time is it?
StartTime := time.Now()
// Spin off a goroutine that just locks protected and waits
go func() {
// Lock it
protected.secure.Lock()
// Tell us when it was locked
fmt.Println("Locked at " + time.Since(StartTime).String())
// Create a timer
WaitForIt := time.NewTimer(time.Minute * 1)
// After the timer elapses
<-WaitForIt.C
// Unlock the struct
protected.secure.Unlock()
// All done
return
}()
// Another goroutine will try writing a value to firstname
go func() {
// An infinite loop because...why not?
for {
protected.firstname = "John"
}
}()
// This goroutine will write a value to lastname
go func() {
// See the previous routine
for {
protected.lastname = "Doe"
}
}()
// And yet another goroutine does nothing but reads values from protected to the stdout
go func() {
// A couple quick flags
var firstchanged bool = false
var lastchanged bool = false
// Create a ticker to check the status of the struct periodically
ticker := time.NewTicker(time.Nanosecond * 5)
for range ticker.C {
if protected.firstname == "John" && firstchanged == false {
// Did the first name get filled in?
fmt.Println("Firstname changed at " + time.Since(StartTime).String())
firstchanged = true
}
if protected.lastname == "Doe" && lastchanged == false {
// Did the last name change?
fmt.Println("Lastname changed at " + time.Since(StartTime).String())
lastchanged = true
}
if firstchanged == true && lastchanged == true {
// Both are done. Change the running flag.
Stop = true
}
}
}()
// This basically spins wheels until the variable for stopping is set
// Checks every 50 milliseconds to see if we should quit
for Stop == false {
Waiting := time.NewTimer(time.Millisecond * 50)
<-Waiting.C
}
}
What's happening here?
Main() has only a few jobs to perform. It creates a struct holding 2 strings and a mutex, it creates an instance of that struct, it gets the current time, and creates a little flag variable (old habits die hard) and then checks every 50 milliseconds for that variable to become true.
There are also 4 goroutines:
Goroutine(a) locks the mutex in the struct, prints out the time since Main took the current time that the lock was set, then sits for a minute before unlocking it.
Goroutine(b) keeps trying to set the firstname string in the struct to John.
Goroutine(c) keeps trying to set the lastname string in the struct to Doe.
Goroutine(d) checks every 5 nanoseconds whether the firstname and lastname have changed, and if they did, prints the time since main() set the current time that it was changed. When both firstname and lastname are set, goroutine(d) also sets the "Stop" variable telling main() to quit.
Pretty simple. What happens when it's run?
Locked at 27.549µs
Firstname changed at 174.431µs
Lastname changed at 251.301µs
That's...fast. It ran on the order of microseconds. Pretty good indication that goroutine(a)'s lock didn't stop goroutine(b) and (c) from altering the struct.
Test Two
Let's try something a little different.
// How do mutexes work on a struct? Let's see if this works the way I hope it does...
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type ProtectThese struct {
secure sync.Mutex
firstname string
lastname string
}
// Create an instance of the structs
var protected ProtectThese
// And a cheap flag for quitting the main() goroutine
var Stop bool = false
// What time is it?
StartTime := time.Now()
// Spin off a goroutine that just locks protected and waits
go func() {
// Lock it
protected.secure.Lock()
// Tell us when it was locked
fmt.Println("Locked at " + time.Since(StartTime).String())
// Create a timer
WaitForIt := time.NewTimer(time.Minute * 1)
// After the timer elapses
<-WaitForIt.C
// Unlock the struct
protected.secure.Unlock()
// All done
return
}()
// Another goroutine will try writing a value to firstname
go func() {
// Let's wait a little bit before trying to set this.
Pause := time.NewTimer(time.Second * 30)
<-Pause.C
// An infinite loop because...why not?
for {
protected.firstname = "John"
}
}()
// This goroutine will write a value to lastname
go func() {
// See the previous routine
for {
protected.lastname = "Doe"
}
}()
// And yet another goroutine does nothing but reads values from protected to the stdout
go func() {
// A couple quick flags
var firstchanged bool = false
var lastchanged bool = false
// Create a ticker to check the status of the struct periodically
ticker := time.NewTicker(time.Nanosecond * 5)
for range ticker.C {
if protected.firstname == "John" && firstchanged == false {
// Did the first name get filled in?
fmt.Println("Firstname changed at " + time.Since(StartTime).String())
firstchanged = true
}
if protected.lastname == "Doe" && lastchanged == false {
// Did the last name change?
fmt.Println("Lastname changed at " + time.Since(StartTime).String())
lastchanged = true
}
if firstchanged == true && lastchanged == true {
// Both are done. Change the running flag.
Stop = true
}
}
}()
// This basically spins wheels until the variable for stopping is set
// Checks every 50 milliseconds to see if we should quit
for Stop == false {
Waiting := time.NewTimer(time.Millisecond * 50)
<-Waiting.C
}
}
What's happening here?
Not much has changed; I just altered goroutine(b) to have a 30 second pause. What happens when I run it?
Locked at 44.836µs
Lastname changed at 160.966µs
Firstname changed at 30.000173283s
Shows that the timers are working. Still shows the mutex isn't doing anything.
Test Three
Time to try wrapping one of the goroutines in a mutex lock.
// How do mutexes work on a struct? Let's see if this works the way I hope it does...
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type ProtectThese struct {
secure sync.Mutex
firstname string
lastname string
}
// Create an instance of the structs
var protected ProtectThese
// And a cheap flag for quitting the main() goroutine
var Stop bool = false
// What time is it?
StartTime := time.Now()
// Spin off a goroutine that just locks protected and waits
go func() {
// Lock it
protected.secure.Lock()
// Tell us when it was locked
fmt.Println("Locked at " + time.Since(StartTime).String())
// Create a timer
WaitForIt := time.NewTimer(time.Minute * 1)
// After the timer elapses
<-WaitForIt.C
// Unlock the struct
protected.secure.Unlock()
// All done
return
}()
// Another goroutine will try writing a value to firstname
go func() {
// Set the lock here in addition to the previous goroutine
protected.secure.Lock()
// An infinite loop because...why not?
for {
protected.firstname = "John"
}
// Unlock
protected.secure.Unlock()
}()
// This goroutine will write a value to lastname
go func() {
// See the previous routine
for {
protected.lastname = "Doe"
}
}()
// And yet another goroutine does nothing but reads values from protected to the stdout
go func() {
// A couple quick flags
var firstchanged bool = false
var lastchanged bool = false
// Create a ticker to check the status of the struct periodically
ticker := time.NewTicker(time.Nanosecond * 5)
for range ticker.C {
if protected.firstname == "John" && firstchanged == false {
// Did the first name get filled in?
fmt.Println("Firstname changed at " + time.Since(StartTime).String())
firstchanged = true
}
if protected.lastname == "Doe" && lastchanged == false {
// Did the last name change?
fmt.Println("Lastname changed at " + time.Since(StartTime).String())
lastchanged = true
}
if firstchanged == true && lastchanged == true {
// Both are done. Change the running flag.
Stop = true
}
}
}()
// This basically spins wheels until the variable for stopping is set
// Checks every 50 milliseconds to see if we should quit
for Stop == false {
Waiting := time.NewTimer(time.Millisecond * 50)
<-Waiting.C
}
}
What's happening here?
This time in addition to goroutine(a) setting the mutex, I added the mutex around goroutine(b) as well. This is not entirely safe in that there is a race condition; it appears that goroutine(a) is spun up faster than (b), so it sets the lock sooner in every test I've run. If something were to delay (a) running, (b) could get to it first. Just something to remember.
What happens when this one is run?
Locked at 23.15µs
Lastname changed at 114.786µs
Firstname changed at 1m0.000142691s
Ah-ha! Goroutine(b) is stuck until the lock set by (a) is unlocked! This confirms that the mutex isn't magically wrapping the struct and set by whatever calls it...it's more like a flag, and any code that might interfere with it must still be mindfully set to check that flag before altering things. In other words, if you want the struct to be protected, you still have to place checks in your code where the code might stomp on the state of the struct; the mutex doesn't lock the struct from random changes.
Some of the sources I found online could be interpreted as saying that the mutex only locks the data right under it (that seems strange?) in the declarations, which here would mean firstname is protected, but lastname isn't. Let's test that.
Test Four
// How do mutexes work on a struct? Let's see if this works the way I hope it does...
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type ProtectThese struct {
secure sync.Mutex
firstname string
lastname string
}
// Create an instance of the structs
var protected ProtectThese
// And a cheap flag for quitting the main() goroutine
var Stop bool = false
// What time is it?
StartTime := time.Now()
// Spin off a goroutine that just locks protected and waits
go func() {
// Lock it
protected.secure.Lock()
// Tell us when it was locked
fmt.Println("Locked at " + time.Since(StartTime).String())
// Create a timer
WaitForIt := time.NewTimer(time.Minute * 1)
// After the timer elapses
<-WaitForIt.C
// Unlock the struct
protected.secure.Unlock()
// All done
return
}()
// Another goroutine will try writing a value to firstname
go func() {
// Set the lock here in addition to the previous goroutine
protected.secure.Lock()
// An infinite loop because...why not?
for {
protected.firstname = "John"
}
// Unlock
protected.secure.Unlock()
}()
// This goroutine will write a value to lastname
go func() {
// Set the lock
protected.secure.Lock()
// See the previous routine
for {
protected.lastname = "Doe"
}
// Unlock
protected.secure.Unlock()
}()
// And yet another goroutine does nothing but reads values from protected to the stdout
go func() {
// A couple quick flags
var firstchanged bool = false
var lastchanged bool = false
// Create a ticker to check the status of the struct periodically
ticker := time.NewTicker(time.Nanosecond * 5)
for range ticker.C {
if protected.firstname == "John" && firstchanged == false {
// Did the first name get filled in?
fmt.Println("Firstname changed at " + time.Since(StartTime).String())
firstchanged = true
}
if protected.lastname == "Doe" && lastchanged == false {
// Did the last name change?
fmt.Println("Lastname changed at " + time.Since(StartTime).String())
lastchanged = true
}
if firstchanged == true && lastchanged == true {
// Both are done. Change the running flag.
Stop = true
}
}
}()
// This basically spins wheels until the variable for stopping is set
// Checks every 50 milliseconds to see if we should quit
for Stop == false {
Waiting := time.NewTimer(time.Millisecond * 50)
<-Waiting.C
}
}
What's happening here?
Both goroutines(b) and (c) are wrapped in the call to the mutex lock. Output from the application:
Locked at 24.157µs
Firstname changed at 1m0.000158927s
^C
Ooh. I had to control-C that. What happened?
The key probably lay in here:
// Spin off a goroutine that just locks protected and waits
go func() {
// Lock it
protected.secure.Lock()
// Tell us when it was locked
fmt.Println("Locked at " + time.Since(StartTime).String())
// Create a timer
WaitForIt := time.NewTimer(time.Minute * 1)
// After the timer elapses
<-WaitForIt.C
// Unlock the struct
protected.secure.Unlock()
// All done
return
}()
// Another goroutine will try writing a value to firstname
go func() {
// Set the lock here in addition to the previous goroutine
protected.secure.Lock()
// An infinite loop because...why not?
for {
protected.firstname = "John"
}
// Unlock
protected.secure.Unlock()
}()
Goroutine(a) locks the struct, waits a minute, then unlocks and quits. The goroutine is gone.
Goroutine(b) then gets the struct next; it locked the struct, then enters an infinite loop setting the firstname repeatedly. The unlock() is never called!
Let's try fixing that.
Test Five
// How do mutexes work on a struct? Let's see if this works the way I hope it does...
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type ProtectThese struct {
secure sync.Mutex
firstname string
lastname string
}
// Create an instance of the structs
var protected ProtectThese
// And a cheap flag for quitting the main() goroutine
var Stop bool = false
// What time is it?
StartTime := time.Now()
// Spin off a goroutine that just locks protected and waits
go func() {
// Lock it
protected.secure.Lock()
// Tell us when it was locked
fmt.Println("Locked at " + time.Since(StartTime).String())
// Create a timer
WaitForIt := time.NewTimer(time.Minute * 1)
// After the timer elapses
<-WaitForIt.C
// Unlock the struct
protected.secure.Unlock()
// All done
return
}()
// Another goroutine will try writing a value to firstname
go func() {
// Set the lock here in addition to the previous goroutine
protected.secure.Lock()
// Loop to set the firstname
for protected.firstname != "John" {
protected.firstname = "John"
}
// Unlock
protected.secure.Unlock()
}()
// This goroutine will write a value to lastname
go func() {
// Set the lock
protected.secure.Lock()
// See the previous routine
for protected.lastname != "Doe" {
protected.lastname = "Doe"
}
// Unlock
protected.secure.Unlock()
}()
// And yet another goroutine does nothing but reads values from protected to the stdout
go func() {
// A couple quick flags
var firstchanged bool = false
var lastchanged bool = false
// Create a ticker to check the status of the struct periodically
ticker := time.NewTicker(time.Nanosecond * 5)
for range ticker.C {
if protected.firstname == "John" && firstchanged == false {
// Did the first name get filled in?
fmt.Println("Firstname changed at " + time.Since(StartTime).String())
firstchanged = true
}
if protected.lastname == "Doe" && lastchanged == false {
// Did the last name change?
fmt.Println("Lastname changed at " + time.Since(StartTime).String())
lastchanged = true
}
if firstchanged == true && lastchanged == true {
// Both are done. Change the running flag.
Stop = true
}
}
}()
// This basically spins wheels until the variable for stopping is set
// Checks every 50 milliseconds to see if we should quit
for Stop == false {
Waiting := time.NewTimer(time.Millisecond * 50)
<-Waiting.C
}
}
What's happening here?
Just added a quick check in the goroutines that set firstname and lastname so once the variable has the value set, they unlock and quit. Running the application yields the following output:
Locked at 30.974µs
Firstname changed at 1m0.000698523s
Lastname changed at 1m0.000794631s
That seems to show that yup, it locks for both those values in the struct.
So at this point it seems that you can lock the whole struct by just have a sync.Mutex included, and it's safe from alterations from code that you specifically wrap in a call to lock and unlock; if you forget, though, code that isn't wrapped in a fuzzy lock() blankie will merrily alter the values of your struct without batting an eye.
One last thing...is code trying to do something to the struct blocked, like channels get blocked waiting for a message to be pulled? Or do they try repeatedly? What does goroutine(b) do as it's currently written?
Test Six
// Another goroutine will try writing a value to firstname
go func() {
// Announce what I'm doing
fmt.Println("About to lock struct so I can change firstname at " + time.Since(StartTime).String())
// Set the lock here in addition to the previous goroutine
protected.secure.Lock()
// I locked it!
fmt.Println("Locked struct to change firstname at " + time.Since(StartTime).String())
// Loop to set the firstname
for protected.firstname != "John" {
protected.firstname = "John"
}
// About to unlock
fmt.Println("About to unlock struct for firstname at " + time.Since(StartTime).String())
// Unlock
protected.secure.Unlock()
// And done...
fmt.Println("Finished with altering firstname at " + time.Since(StartTime).String())
}()
What's happening here?
All I've changed is goroutine(b) so it has some additional announcements to STDOUT. Here's what the new output says:
Locked at 25.739µs
About to lock struct so I can change firstname at 121.747µs
Lastname changed at 1m0.000195611s
Locked struct to change firstname at 1m0.000158478s
Firstname changed at 1m0.000593511s
About to unlock struct for firstname at 1m0.000590839s
Finished with altering firstname at 1m0.000960257s
To me this shows that the mutex blocks access; the process just waits. But that could be because of the structure of the code; let's try constantly hammering it in a loop again.
Test Seven
// Another goroutine will try writing a value to firstname
go func() {
// Loop to set the firstname
for protected.firstname != "John" {
// Announce what I'm doing
fmt.Println("About to lock struct so I can change firstname at " + time.Since(StartTime).String())
// Set the lock here in addition to the previous goroutine
protected.secure.Lock()
// I locked it!
fmt.Println("Locked struct to change firstname at " + time.Since(StartTime).String())
// Make the change
protected.firstname = "John"
// About to unlock
fmt.Println("About to unlock struct for firstname at " + time.Since(StartTime).String())
// Unlock
protected.secure.Unlock()
// And done...
fmt.Println("Finished with altering firstname at " + time.Since(StartTime).String())
}
}()
What's happening here?
Once again it's a relatively minor change in goroutine(b); now everything is encased in a loop instead of just the "let's make a change" bit. What is the output?
Locked at 36.342µs
About to lock struct so I can change firstname at 155.914µs
Lastname changed at 1m0.000200095s
Locked struct to change firstname at 1m0.000167036s
Firstname changed at 1m0.001612157s
About to unlock struct for firstname at 1m0.001605558s
Finished with altering firstname at 1m0.003361782s
Still blocking while the lock is set; it's not constantly retrying the loop or the "About to lock" message would keep repeating. I'm fairly confident that if you have a second process trying to do something on a struct you've locked from another process, it'll just block until the mutex is unlocked!
One last bit of fun, just to see what effect it would have; I wrapped the routine that checked for the status of the struct in a lock vs. no lock to see how many times it checked (after lowering the lock from goroutine(a) to 5 seconds);
// And yet another goroutine does nothing but reads values from protected to the stdout
go func() {
// A couple quick flags
var firstchanged bool = false
var lastchanged bool = false
// Create a ticker to check the status of the struct periodically
ticker := time.NewTicker(time.Nanosecond * 5)
for range ticker.C {
// Add one to the counter
counter = counter + 1
// Lock it
//protected.secure.Lock()
if protected.firstname == "John" && firstchanged == false {
// Did the first name get filled in?
fmt.Println("Firstname changed at " + time.Since(StartTime).String())
firstchanged = true
}
if protected.lastname == "Doe" && lastchanged == false {
// Did the last name change?
fmt.Println("Lastname changed at " + time.Since(StartTime).String())
lastchanged = true
}
if firstchanged == true && lastchanged == true {
// Both are done. Change the running flag.
Stop = true
}
// Unlock it
//protected.secure.Unlock()
}
}()
You can see the part I commented out, of course, to enable to lock. The results were kind of interesting (I assume it's because the check just blocked for the vast majority of 5 seconds...)
Without the lock: "I checked the status of the struct 124 times."
With the lock: "I checked the status of the struct 222896 times."
Maybe something to keep in mind if you have mutexes, a number of goroutines hitting excluded data, and performance issues!
