On a bit of a personal note, I just realized how long it's been between blog posts. Over two months! Wow!
I didn't die. I managed to avoid death; instead I have been recovering from a couple of broken bones. As I told a coworker at one point, I definitely do not have a superhuman ability to stop cars with my body.
That's a quick few lines explaining my lack of recent updates; obviously it has little to do with the title of this post. But on to the point. Well, sort of. This is tagged as a <rant>, not a well structured essay. If stream-of-thought writing isn't your thing, this is another post to avoid.
I've worked for several years in the public school system in the past. It was my first "real" job outside of working in a small ISP/computer repair business and served as what would become my introduction to the real world of politics in small town schools. (I should note that my direct experiences are in relatively rural PA schools; laws vary in other states dictating the specifics of how schools are run. Also these are my own experiences. I have family that have been involved in the local schools for combined decades of experience...this particular case reflects things that happened to or around me during my time in those trenches.)
I remember becoming very frustrated with leadership in schools. I worked in the IT department in system where hundreds of people were working in a hierarchy charged with taking in ignorant kids and creating educated citizens at the other end of a pipeline. You had a (relatively) large number of kids being led by a group of teachers who were in turn led by principals who were then led by a superintendent, and auxiliary services such as maintenance and supplies have another branch of leadership.
It's a format that should be a natural case study in leadership 101. The goals are clearly laid out. Superficially, results should be simple to measure (are the kids graduating because they meet actual goals or because they've been in the system a particular number of years?), and outcomes trackable. Information and status can be sent up the line, and directives and initiatives are passed from the superintendent and principals back down the line, leading to more measuring of outcomes and evaluating the level of success.
Schools even reflect (and reinforce) this idea. You have a chain of command where grievances are filed and issues heard. There are numerous meetings among departments and grade levels that interact with the management level leaders. Standardized testing is now, more than ever, ingrained into the public school culture.
But I came to realize that the schools didn't really do anything in terms of innovating the culture of learning. Classrooms are largely structured the same as they had been for decades; allowing for changes in fashion and substitute high-speed printers in place of mimeograph machines, the classroom setting is more or less recognizable to today's students. School learning paradigms have become nearly universal blocks of pre-packaged tripe...the school structure itself hasn't changed (teachers are lecturing at the front of the class, bells signal shuffling to or changing of the next subject, etc.).
Indeed it seems the only things that have substantially changed is the terminology encapsulating the latest magic solution to education problems (No Child Left Behind! Mainstreaming! Outcomes Based Education!) and expectations laid at the feet of educators by the public (it's hard not to find evidence that schools are regarded as free babysitting for an increasingly strained middle- to low-class worker in the community...)
Why is that? I was largely blind to the reality that schools themselves are not so much looked to as innovators as much as they are maintainers of status quo; opportunities for innovation weren't just ignored, but actively discouraged.
Teachers with a well of knowledge built over years of practical experience could probably do some interesting curriculum work in their subject area. Today the means of creating a kind of custom curriculum...tests, handouts, worksheets, books for a given subject area...is completely within the reach of even the most modestly equipped classroom. What I've seen is teachers encroaching on that idea being discouraged if they deviate from the expected curriculum supplied to them.
Rather than being empowered to teach using experience and new methods, then evaluated on outcomes, teachers are beholden to parents, regardless of what qualifications the parent has in judging any aspect of the classroom experience. Teaching is now a minefield in potential disasters, and the leadership in no way leads with authority when a parent is in some way offended. Soft subjects such as English are an especially ripe subject for criticism when parents aren't bright enough to understand parody, double entendre or sarcasm, the use of which was a staple for writers such as William Shakespeare and Jonathan Swift.
The baseline ignorance only seems beneficial when the community is too dim to fully grasp the bawdy humor of Shakespeare...unfortunately the moment a student's learning leads to piercing the veil of ignorance, should they share with their parents the discovery that a nunnery is not in fact for housing nuns in the presumed context, the teacher is once again found to be hovering a foot over a landmine.
The true leader is someone who has faith that his or her staff are capable of doing their jobs. They guide and nurture when necessary, and allow freedom to fail. Problems are issues to be rectified, not cancers to be cut out from the organization, unless all other methods to correct issues have failed. And leaders...good managers...shield the people in their charge from shit raining down from above. They pass goals and edicts that are necessary for their people to do their jobs while "up-managing," with authority, when people outside their silo try to swipe at their people. They do this because the organization is good and wants what is best for the common good.
It was only after being in the environment for over a decade that I understood you don't see innovation in public schools because they do not have true leadership; they instead are an agency that act as a government entity, only separate enough to not be directly governed by or have accountability to higher governance, but still beholden to actual state- and federal- rules. It's a strange hybrid that evolved from the original model for public education: the factory.
Today's schools are ostensibly self contained entities with day to day operations ruled by the principals in each building and business for the district answering to the superintendent. Teachers are constantly monitored by administrators, many of whom have classroom experience from the distant past but still weigh in on minutiae of classroom experience by sitting in on short periods of instruction and extrapolating from these sessions what an entire class is like, even if they have little to no certification in that subject matter.
Above the administrators is a school board, a group of community members elected by a small percent of the community to conduct school business and make sure administrators are acting in the best interest of the learning community.
Beyond this group is the state and federal education departments; they supply funding and issue mandates for recognition of state accreditation. These groups don't normally directly interact with schools, leading to reinforce the idea that schools are a quasi-government agency. Their influence is not unfelt, however; if schools want a large percentage of their funding, they have to follow certain rules dictated by the government. For example, schools don't filter Internet traffic because it's beneficial for protecting students; it's required if they want funding for Internet access and/or equipment, despite the overhead added in the burden on staff since students are constantly trying to find new proxy sites or bypass filters using their cellphones as access points and as of the time I left the school system the funding mandating filters on student access was not to be used to filter student access. The funding mandates (also called unfunded mandates within the school system) had stipulations often unrelated to what they mandated; it's like requiring a kid to eat all their peas at dinner if they are to be allowed to watch a favorite program on TV.
Schools themselves end up operating as a kind of system maintaining the status quo; the state hands them mandates, sometimes in the form of laws, sometimes in the forms of conditions that withhold the school's allowance if they don't comply. Parents bully teachers into creating safe spaces that conform to their own form of "safe space," creating a lowest-common-denominator education bar (heaven help you if you promote witchcraft by expressing an interest in Harry Potter! And a sex reference in a book? Never allowed...by the way, Jay Asher's Thirteen Reasons Why exposes kids to the idea of suicide, and we'd never want to have kids thinking about that, so we should ban that as well!)
The people I thought were leaders in the school organization instead buffer themselves from the responsibility of leading by complying with the vocal minority of the community. Teachers must navigate the minefields of teaching with little to no support from principals because they tend to act more in the interest of avoiding complaints than creating an environment of academic freedom. Often this is justified as conforming to "community standards" and enforced by the school board, although in my experience the school board often pays attention more to personal issues than school issues as a whole (one funny example was when the board okayed a teacher's request to pay for training on a particular skillset and at the same meeting vote to eliminate the class where that skill would have been used by that teacher.) This generally leads to a system where complaints are addressed, but positive reinforcement is rare.
What is the positive reinforcement? The general attitude I see today is "you have a job." Administrators don't rock the boat, and the teachers, most vulnerable to landmines because they have to please the administrators above them without angering the parents of dozens of students, are under constant stress to "perform" without challenging students (well, parents by proxy...) ideology; often this is an impossible task. For example, the Bible is, technically speaking, a book of myths in terms of literature. But if you discuss this in a public school using actual academic terminology, you can bet your ass the parents will scream. They're typically okay if you apply the same terminology to a Quran and you have no Muslims in the community, though, even though it's the same idea.
Administrators fixate on enforcing mandates; the highest levels of school management are actually middle management. The community, often without understanding the system, are direct policy if it's not covered by state mandates (and who better to judge education than wealthy male politicians with no education experience?) This is a recipe for a quagmire.
Schools can't act on their own or take an initiative for innovation. This is why schools turn to other schools for "best practices"; to do something on their own means they take responsibility for their actions. Everything must be presented as "how they do things at $NEIGHBORING_SCHOOL_DISTRICT;" that way if it fails, it's because of something the teachers did wrong when implementing the directive. Administrators avoid anything that would displease the school board for fear of being fired, and when the school board and community are most concerned with "results" and "tax money", that means spending large amounts of time making sure blame is pushed down on the teachers for not following directives properly (vague directives with specific goals are their friends in these cases.)
This structure also leads to the large sums of money spent on consultants and presenters. I was amazed at the amount of cash doled out to people brought in over in-service days to tell teachers how to teach the latest fad; while I wasn't in the system as long as some people I knew, I was in it long enough to see a couple of "education shifts" (the changing of fads were rarely acknowledged when they failed. They simply slid away as they were overshadowed by the Next Big Thing.) Some administrators...usually with a title vaguely tied to curriculum management...would hear about a new trend at a conference and justify their usefulness by lining up speakers to impart wisdom to the teachers (because they spent years in college learning how not to teach, I suppose.)
I suppose there's irony in that if the curriculum person didn't do this, they'd be considered not doing their jobs.
Create an environment where the leadership spends a good chunk of time catering to vague standards perceived to be community preferences (often driven less by educational goals and more by personal agenda) with a heaping helping of blame-shifting, strain out accountability at the administrative level, and you get a climate that breeds management by personal vendettas.
In most small businesses and startups it's said that the founders set the personality of the business. Apple is famous for the Steve Jobs "founder DNA" after the departure of the Woz and the rise of coverage regarding diversity issues have given subtle hints of the leadership style of several tech companies. But leadership in public schools are rarely remembered by people aside from their staff, and it's rarely a scale of how "good" someone is. More often than not it's a scale of how incompetent they were in managing his or her staff.
The scale of success is often weighed against the perceived damage they'd done (or avoided). Previous superintendents and principals are rarely recalled fondly except in the context of why the current leadership is inadequate in properly handling issues. It's hard to tell the number of times I've heard some variation of, "This would never have happened if $NAME were still here..."
But that's only in handling personnel issues. In terms of lasting legacy, there are two ways for a person in some position of power to assert him or herself; changes in staffing, and major projects. The staffing changes are mostly internal recognition. Usually this is where realization of nepotism takes place; often candidates interviewing for positions then getting job offers just happen to have graduated from that school district. This tends to be relatively benign...at least they can justify this as cultivating from within the community, and students who become staff are essentially the school reaping what it sows.
On the other hand there are people who come into positions from farther outside the district and start hiring relatives or candidates who are entirely outside the community. Not in terms of a diverse hiring pattern from all around the state, but rather constantly importing from the previous employer, like situations in which an angry ex-employee from one company gets hired at a competitor and the first thing they do is start soliciting their division to follow suit. Not only is this transparent to existing employees but calls into question if the new coworkers are there because they are qualified or because they had a keyword in the resume.
The second method of legacy is construction projects. These are most visible to the community and take careful mastery in politics to pass, as large scale renovation or addition of a facility is an obvious show of spending taxpayer money, which is an obvious trigger for the school board to reflexively hate the administrator in question.
What these both have in common: politics. Since the leadership isn't actually leading with ideas for improving the state of education or education initiatives, the political landscape become internal competitions, a teacher's Hunger Games of avoiding ill-attention from principals lest you become a target for culling.
Looking back I think I came to feel that the education environment, while broken in many ways as a property of the system, was made toxic by those in leadership roles. When the superintendent was decent...not just at doing the job, but a decent human being with decent goals that bordered on being overly idealistic at times, given the limitations imposed by the system...the stressors would come more often than not by the willingness of administrators to automatically bend to the arbitrary will of parents or from the school board (not coincidentally around the time new contracts were due, as that was the time when the school board would make it abundantly clear that they do not value educators in their community.) But when new leaders move into position with their own vendettas, sagging morale would move from bending to nearly breaking. Teachers are put into a position of having to navigate the ideology minefield of student (parent) expectations AND having to avoid giving administrators a target on which to push them out of a job and replace them with less experienced (and thus easier to manipulate and give lower pay) hires of their own preferences, and in some cases preferential hiring would start a legacy of staff with obligational loyalty.
I thought at first this type of behavior was anomalous until a friend in another state related a similar story of this happening to his sister when she worked in public education; her experience made her quit the profession altogether.
What does this have to do with innovation (or lack thereof) in public schools? There's no incentive to innovate. The rewards for management comes from exerting limited power over their spheres of influence (hiring/firing people in exchange for proper loyalties, for example) or pushing for projects of limited usefulness (Another experienced example: grants for a computer lab grab a local headline and benefits schools for about 2 years, after which the equipment tends to fail due to lack of maintenance funding.) Changes are slow, and only implemented if there are volumes of justification from neighboring school districts doing something similar and if blame for failure can be easily shifted to underlings. Even if teachers have an idea for an improvement, they risk putting a target on their backs (you can't shift blame any lower than teacher on the hierarchy) and they typically have an overloaded schedule as it is from understaffing.
I guess the most embarrassing part is the amount of time it took to realize that the public school system almost purposely lacks incentive to innovate. Instead it has become an environment that incentivizes a CYA principle (Cover Your Ass), where the best course of action is to keep your head down and stay off the radar of others. I've rarely witnessed administrators reward, but appearing when there were problems became so commonplace that it became a common meme; if a teacher saw an administrator, it was because there was a problem. What started as a factory model for efficiently taking children and indoctrinating them into citizens benefiting society is a quagmire of looking out for oneself, with little to no sense of true camaraderie at any level.
The saddest part is that because this is a system, change will be very, very slow. Attempts to create an innovative system...think charter schools or alternative schools...are usually blocked, because new schools must jump hurdles for accreditation but also drain funding from public schools in the district where the new school is proposed, so they get to have a say in whether the charter school is opened. If the fact that schools depicted in older movies are so very familiar to today's audiences, this probably isn't a sign that change will be welcome anytime soon.
Sunday, November 22, 2015
Thursday, September 10, 2015
Adventures in Medication
As processes get more complex, there is a tendency to try automating and streamlining the process. Usually when something goes wrong in that automated chain, humans intervene, and whatever one-off issue is handled by some form of intelligence and life moves on.
Apparently this doesn't necessarily apply to healthcare. At least not in this case. And yet again I get confirmation bias that when insurers talk about the need for onerous copays so people take responsibility for their healthcare decisions, it's deflective and disingenuous.
I had an appointment with my doctor. My blood sugars are still off, and she reiterated a need to lose weight and lower the blood sugars. But this time she said there's a drug that shows promise in lowering blood sugars without weight gain as a potential side effect. As a matter of fact, one of the side effects is weight loss.
I'd asked her on a previous visit about the possibility of getting an appetite suppressant. At the time she offered a fat blocker. If you know what a fat blocker does, you can imagine the potential side effects for someone who has an hour commute through the subways and sidewalks of the city are not pleasant. So...no.
So this new medication, called Victoza, was good news to me. Oh, sure, there's some potentially (potentially) horrible side effects, but so does obesity and high blood sugar. She had her office call in the prescription on Monday.
A day later the prescription showed up in my pharmacy's system. But unlike the other prescriptions the office renewed, which were ready for pickup when I checked, the Victoza was showing up as "on hold" due to insurance issues.
"I'll give it another day, they're probably having to get it authorized." Although I did notice that the Victoza was not called Victoza. It was called Saxenda.
I wasn't too worried, though. See, Saxenda and Victoza are just other names for Liraglutide. Medications are substituted all the time for equivalents, so I didn't think anything of it.
That is, until I gave it a little more time and the hold was still on the medication. I complained about this on Twitter and my insurance company replied with a suggestion to email their "let us help you" line. I emailed them the details of what was happening.
Their rep looked into it and said our plan doesn't cover obesity medication.
"Uh...this was prescribed for diabetes," I said. I also made a half-snarky query regarding whether injectable insulin was covered.
In the meantime I sent a message to the doctor's office relaying what the insurance company was saying. The response I get back is something about the issue being sent to some department that handles insurance claims or...interaction...something. They handed it off to a group whose job is dealing with insurance, I think.
So now I have the insurance company help people looking at the issue and I'm told the hospital is looking into it.
The insurance people get back to me, telling me that insulin is covered under the plan. He also said that the Saxenda being using for diabetes treatment might be possible if the doctor tried submitting an authorization to use it for that purpose. "The information I have is indication(sic) Saxenda as an anti-obesity drug, which is not covered."
I am finding it amusing at this point that they have no qualms about covering diabetes medication but treatments to try lowering weight is not covered, when I have little doubt that the cost of the side effects of obesity are probably more expensive by several factors.
I then talked to our HR person who directed me to a contact with the third party benefits management company that acts as our liaison with the insurance company. After some back and forth, she said that "...what it comes down to is that Saxenda is not on their formulary list of covered medications. Every insurance carrier has a formulary and although one particular drug is covered another one in the same drug class may not be covered. When we pushed back to our rep to have this reviewed again she sent us an excerpt from the Saxenda website which states this is a weight loss drug, and it also states that it is not for the treatment of type 2 diabetes. This being said, Cigna does not cover weight loss medications so that is the reason they would not cover Saxenda under any circumstances."
This...was strange. They're going by website copy for the drug?
Keep in mind that Saxenda is another name for Liraglutide. Liraglutide is another name for Victoza. Exact same drug...press releases for Saxenda don't hide this fact. See, the manufacturer of Victoza, Novo Nordisk, noticed that people taking Victoza were losing weight at a more-than-coincidental rate. So Novo took Victoza to the FDA and had the drug evaluated for weight loss under the marketing name Saxenda. Same drug. Different name. After trials, Saxenda was approved as a weight loss drug; the only difference I could find in any of the literature was the dosage.
So they're rejecting the drug because...it's showing as Saxenda?
It turns out Saxenda is listed as a "for fatties" drug while Victoza is listed as a "for diabetics" drug. Again, the only difference is the name.
But now I notice they are entirely focused on Saxenda. Not Victoza. I message the doctor's office and they verify that the prescription was in fact for Victoza. Also the doctor's office sent a prescription for Lantus since the insurance company continued to deny coverage.
Another side note; the messages from the third party management company and insurance company both expressed regret that I didn't get the news I hoped for. I assume that this is a polite way of appearing to care.
Shortly after that message I get another from the insurance company helpline explaining that he was working from the RX number I provided in my initial emails.
Now I know that:
- The doctor wrote a prescription for Victoza
- The pharmacy is trying to authorize Saxenda
- The pharmacy is using an RX number that somehow maps to Saxenda when it's referenced
- Everyone involved is literally going by the prescription name Saxenda with no regard to what Saxenda is
- They believe the information literally given by the promotional website
I open the Victoza website and send the reps the following cut/paste:
What is Victoza®?
Victoza® is an injectable prescription medicine that may improve blood sugar (glucose) in adults with type 2 diabetes, and should be used along with diet and exercise.
I asked if this was covered under our drug plan, and the insurance company and third-party company both tell me that yes, Victoza is covered under our company health plan. My eyes couldn't possibly roll back any harder.
I asked the insurance company what RX number my doctor would have to use in order to get that particular medication prescribed. They reply that she can't; "The RX number can change depending on the manufacturer of medication and pharmacy you go to. Even a particular pharmacy may change their RX numbers from time to time. So you would not want to use the RX number to fill a prescription, you would want to use the drug name specifically."
Now I know that while they can refer to the RX number and map it to a particular drug at that pharmacy, the numbers apparently aren't static.
The third-party management company, when posed the same question, said: "It just has to be a Prescription for Victoza and they should not include DAW (dispense as written) so the pharmacy can fill it with the generic if there is one available."
I pass this information to the doctor's office...again...and they send in the prescription. This time it passes through their system without any problem.
It took me a week...actually closer to a week and a half...to get the prescription filled.
This adventure had a hospital, insurance company, a third party management company, and possibly the pharmacy all involved in sorting this mess out, and it appears no one but me knew (and the doctor) that Saxenda and Victoza were names for Liraglutide. A two minute Google search would have told them this, and yet it didn't occur to anyone to say, "Hey, the patient keeps talking about diabetes, and Saxenda with the name Victoza is the diabetes version of the drug! We can probably pass it through the system without any problems!"
I can't help but feel that I did the work that people in three companies over the course of a week and a half were supposed to do in less time.
This is why I have come to believe the whole "patient taking responsibility" thing is bullshit. There is no way the average person could be reasonably expected to know things like how RX numbers are mapped to drugs that may or may not be referenced (I would have thought they were a type of serial number...apparently it depends entirely on the pharmacy and manufacturer, and they can change without warning. Not. Useful.)
Even when I was trying to sort out what was going on, it was like the left hand didn't know what the right hand was doing. It didn't occur to anyone in this chain to figure out Liraglutide was the drug I was supposed to get? No one could work with the doctor to figure out what was needed?
I invested a lot of time trying to get this sorted out. I'm not a medical professional. I don't know how all these different parts in the grand scheme of healthcare work. And yet these are the organizations that say the patient needs to take responsibility for healthcare decisions (read: make the cost cheaper for the insurance company). They were paying all these people to do something I ended up solving for free (except for my time that I'll never get back.)
Worse, as I tried not to slip into despair at how hopeless it felt to deal with these companies that all sounded like they wanted nothing to do with me after the indication that "we don't deal with fattie drugs, fattie," I can tell the typical patient would just give up. The doctor's office issued a new prescription for a different type of drug (Lantus is insulin, which tends to lead to weight gain...) and as far as I could tell that was the end. That is how I felt in dealing with the mechanisms of the system. The doctor's office and insurance companies routinely do this dance; who was I to fight the system? They're experienced in this matter, and an "oh well, try this drug and see if the insurance company okays it" approach is apparently par for the course.
There is simply no way for the patient to make reasonable decisions in healthcare without being an expert in how all the pieces work. The simple brushoff with...for lack of a better term...victim blaming is a disingenuous mask to hide a broken system and shove responsibility away from the real bad actors. I highly recommend people look into Steven Brill's Bitter Pill article, or his followup work, for more insight on how the healthcare system screws everyone over.
Friday, September 4, 2015
Golang Web Application and MSSQL Injection Attacks
Not long ago I wrote a post on the first steps in developing an application that talks to an MSSQL database.
Those examples worked, obviously. I was compiling and testing the application before posting information. The example was my test case for a package that was later integrated into an application that integrated a web application with database access; that was when someone pointed out something that I should have checked earlier, but hadn't.
Namely in this line:
In this statement there is a bit of content that comes from a user and is added to the database (namely strContent). Because it is concatenated, the string passed can be either a parameter or part of a query; a mischievous user could send commands to alter the database rather than just a value to add to the table.
Whoopsie!
I'll leave it up to you to Google what SQL injection attacks are. There has been a metaphorical ton of digital ink spilled discussing that topic. The summary is they're bad, and they're a very basic mistake in making a web app.
The good news in my case is that for what I was doing, the strContent string was partially filtered; before it was hitting the database, it was being fed into the template library. The template library made the text safe for HTML rendering, so much of the punctuation needed to turn that data into part of a query was transformed into HTML punctuation.
That's kind of a leather armor defense against injection attacks. The next step is to upgrade to chain mail.
The next thing to do is parameterize the query. This makes sure the query doesn't treat that string as part of the query to execute, parameterizing encapsulates the string and protects the database from clever users.
In Go, parameterization is characterized by:
db.Query("SELECT name FROM users WHERE age=?", userinput) // OK
db.Query("SELECT name FROM users WHERE age=" + userinput) // BAD
I altered the sample entry by doing the following:
Basically the "?" marks are stand-ins for the variables in the query statement. I'm not sure if that fully secures the application from injection attacks, but I think it is a step in the right direction. If anyone has more information or suggestions, feel free to leave a comment...
Those examples worked, obviously. I was compiling and testing the application before posting information. The example was my test case for a package that was later integrated into an application that integrated a web application with database access; that was when someone pointed out something that I should have checked earlier, but hadn't.
Namely in this line:
// Add a record entry _, err := db.Exec("USE " + strDBName + "; INSERT INTO testtable (source, timestamp, content) VALUES ('" + strSource + "','" + strconv.FormatInt(int64Timestamp, 10) + "','" + strContent + "');")
In this statement there is a bit of content that comes from a user and is added to the database (namely strContent). Because it is concatenated, the string passed can be either a parameter or part of a query; a mischievous user could send commands to alter the database rather than just a value to add to the table.
Whoopsie!
I'll leave it up to you to Google what SQL injection attacks are. There has been a metaphorical ton of digital ink spilled discussing that topic. The summary is they're bad, and they're a very basic mistake in making a web app.
The good news in my case is that for what I was doing, the strContent string was partially filtered; before it was hitting the database, it was being fed into the template library. The template library made the text safe for HTML rendering, so much of the punctuation needed to turn that data into part of a query was transformed into HTML punctuation.
That's kind of a leather armor defense against injection attacks. The next step is to upgrade to chain mail.
The next thing to do is parameterize the query. This makes sure the query doesn't treat that string as part of the query to execute, parameterizing encapsulates the string and protects the database from clever users.
In Go, parameterization is characterized by:
db.Query("SELECT name FROM users WHERE age=?", userinput) // OK
db.Query("SELECT name FROM users WHERE age=" + userinput) // BAD
I altered the sample entry by doing the following:
// Add a record entry strTimeStamp := strconv.FormatInt(int64Timestamp, 10) _, err := db.Exec("INSERT INTO "+strDBName+" (source, timestamp, content) VALUES (?,?,?);", strSource, strTimeStamp, strContent)
Basically the "?" marks are stand-ins for the variables in the query statement. I'm not sure if that fully secures the application from injection attacks, but I think it is a step in the right direction. If anyone has more information or suggestions, feel free to leave a comment...
Monday, August 31, 2015
Burger King: Have it Your Way, Unless It Take Effort
Burger King has long been known as the chief rival to McDonalds and home to one of the scariest and creepiest mascots in advertising history. But for me, the local Burger King is an amazing anomaly in lessons on running a business.
See, this business has managed to consistently deliver poor service over the years while still staying in business. It became a joke in my family that ordering from the local business was a game in Russian Roulette. Orders never seemed to come out right. There was a streak where I think I had 5 visits and, without fail, they failed to get the order correct. Once I went there and ordered just a soda.
One soda.
They gave me the wrong one.
Really? I ordered one lousy soft drink and you still screwed it up?
At this point most reasonable people would just say, "Don't go there anymore."
To them I say, "No shit?" Because that's what I did. If I was asked where to go for a fast food outing, the local BK was definitely on the bottom of that list. I told people I knew that I despised that place and it was a den of incompetence.
My son is still young enough to not care about such things. "Good enough" meant he had his cheeseburger or croissant sandwich. I'm not sure why he adores the food there. Perhaps part of it is not having to pay actual money to be inconvenienced by details such as, "Do I really want to get up and return this thing I didn't order and ask for my actual order?"
They've gotten better,...but at this point I don't really care.
Sin one: they rarely got my order right. Even simple ones. Like for a single drink.
The other day he was begging to go there for breakfast. It served as a source of more bafflement because I had their sausage, egg and cheese croissan'wich and the sausage tasted burned. Not just grilled...kind of charred on the surface of the patty. But at least it wasn't a piece of charcoal through and through. Edible, but the taste left something to be desired.
Sin two: the food prep doesn't seem very...consistent?
Many years ago...my wife says it was around 2007...my son was with his grandparents when he had an incident at this restaurant. This Burger King has their own play area, with slides and climby parts and...I don't know what else. But my son somehow had a matchbox car drop into a gap in the play area jungle-gym-ish climbing area.
He reached into the gap to retrieve the toy and the plastic pieces pinched together, entrapping him. I found out after everything was said and done and the fire department had left that my son was okay, just a little rattled. I don't think he ever went into the play area again. Yeah...the workers were clueless about what to do, and the fire department helped free him.
He managed to get stuck despite being supervised in a public restaurant playground area. And it was good he was being watched...I shudder to think of what happened if the play area had shifted just right that it could have crunched his arm.
There are certain risks to playing in a playground, and we accept that. But this was more of a maintenance issue. And it created a safety issue. I wanted to contact Burger King and let them know that maybe they might want to be careful about this...you know...inspect their playground equipment once in awhile.
Sin three: child-eating playground equipment.
Contacting Burger King was a challenge, to say the least. They had no Twitter account (Twitter launched in 2006, after all...); I could find no contact on their web page, there was no sign of an email address. The rest of the world had some form of electronic contact. Burger King gave customers an electronic middle finger.
Not kidding. A quick check on the Internet Archive at http://web.archive.org/web/20071006000851/http://burgerking.com/companyinfo/contactus.aspx (late 2007) came right out and said "E-mail communication is not accepted." In 2007. What the hell?
Sin four: about as tech savvy as a three year old, or perhaps K-Mart.
I did something that really just made me angry. I wrote them a letter. An actual, dead-tree, ink on paper letter. I mean, it was printed...I'm not a luddite and I know how to use a word processor. But it still irritated me that asking them (or their franchisees) to have some standard of not smooshing children in their playground equipment really shouldn't cost me money.
What I wanted was an acknowledgement that they'd do something to keep other kids from getting eaten by equipment. What I got was bupkis. I never heard back from Burger King. Not even some offhanded blame against their franchisee, pretending they have no control over how their image and name is represented. Nada. Zip.
My letter went into some great abyss of customer fuck-offs.
Sin five: at least have the decency to acknowledge something like this. There was a fire department called, you bastards.
What got me musing on the myriad reasons I avoid this BK when at all possible is my son's recent insistence...begging, really...that his special day out include a trip to Burger King for morning breakfast. I opened a web browser and it connected to my employer's website. I noticed that there was an oddly shaded and content-empty bar along the bottom of my web page.
Viewing the source showed some issues with loading something from an advertisement link. Which was strange, since my employer doesn't shove any weirdball ads like this at their users (I think it was blanked out because of an ad-block extension active in my browser at the time, rendering whatever their ad bar was supposed to be, blank, instead.)
I opened a new browser tab and navigated to my employer again, but this time used the https: link. That time the advertiser disappeared. See, using an encrypted point-to-point connection makes it difficult to inject extra HTML into my browsing session without me knowing...
...which meant Burger King, the company that for so many years makes it hard to actually contact them with any feedback and displayed a stunning lack of customer interaction and tech savviness, was injecting unwanted ads (and probably monitoring) my web browsing over their wifi.
Sin six: You're monitoring and injecting content into my browsing? Really?
I'm assuming that there was something about this buried in the legalese agreed upon when clicking to use their wifi, although to be honest, I don't remember clicking on anything in order to get on their wifi. And when it comes to public wifi, there's always the danger of your session being hijacked if you're not using a VPN or some other form of encryption. But still...another reason to dislike you?
I'm not sure how you stay in business other than being convenient for people in an area that doesn't have a lot of income. You're close to a retirement home, and getting to the Dunkin' Donuts would mean crossing the street in an area that has horrible crosswalk support. Maybe the Walmart model is at play; cheap food is cheap, so who cares about customer experience?
I even made a passive aggressive complaint on Twitter about their monitoring of web traffic and injecting code into the session (believe it or not, BK is on Twitter now...they joined in 2010. Yeah...little late to the party.) They said nothing. I've made irritated remarks about Dell and Time Warner and had them reply to me without even trying to talk to them. BK doesn't give a damn.
Sin seven: you monitor customer web traffic but not your mentions in Twitter. Get with the program.
Last I went to their website to contact them about the web-jacking. I clicked on the contact us page. It redirects to a "tellusaboutus.com" website.
They outsourced the "contact us."
A totally outside company handles "contact us" for Burger King.
Sin eight: ...I...no. That's enough.
Many companies are at least trying to not suck. They show signs of understanding how to engage with customers. They monitor for signs that customers want to communicate with them, want help, want feedback, want acknowledgement (hey, hear of Netflix? Or Dominoes? Or any of the dozens of other companies that will get in the news with some playful banter on Twitter with customers?)
Most companies at a minimum make it easy to email them with complaints or suggestions. Hell, I've had companies that pester me for feedback via email.
Burger King is like a digital brick wall. They seem to actively not want customer feedback.
...I suppose that kind of explains quite a bit.
I'll look forward to what the next five years holds for Burger King. If McD's is feeling the financial pinch, it may not be long before Burger King topples over. Can't say I'll miss them.
See, this business has managed to consistently deliver poor service over the years while still staying in business. It became a joke in my family that ordering from the local business was a game in Russian Roulette. Orders never seemed to come out right. There was a streak where I think I had 5 visits and, without fail, they failed to get the order correct. Once I went there and ordered just a soda.
One soda.
They gave me the wrong one.
Really? I ordered one lousy soft drink and you still screwed it up?
At this point most reasonable people would just say, "Don't go there anymore."
To them I say, "No shit?" Because that's what I did. If I was asked where to go for a fast food outing, the local BK was definitely on the bottom of that list. I told people I knew that I despised that place and it was a den of incompetence.
My son is still young enough to not care about such things. "Good enough" meant he had his cheeseburger or croissant sandwich. I'm not sure why he adores the food there. Perhaps part of it is not having to pay actual money to be inconvenienced by details such as, "Do I really want to get up and return this thing I didn't order and ask for my actual order?"
They've gotten better,...but at this point I don't really care.
Sin one: they rarely got my order right. Even simple ones. Like for a single drink.
The other day he was begging to go there for breakfast. It served as a source of more bafflement because I had their sausage, egg and cheese croissan'wich and the sausage tasted burned. Not just grilled...kind of charred on the surface of the patty. But at least it wasn't a piece of charcoal through and through. Edible, but the taste left something to be desired.
Sin two: the food prep doesn't seem very...consistent?
Many years ago...my wife says it was around 2007...my son was with his grandparents when he had an incident at this restaurant. This Burger King has their own play area, with slides and climby parts and...I don't know what else. But my son somehow had a matchbox car drop into a gap in the play area jungle-gym-ish climbing area.
He reached into the gap to retrieve the toy and the plastic pieces pinched together, entrapping him. I found out after everything was said and done and the fire department had left that my son was okay, just a little rattled. I don't think he ever went into the play area again. Yeah...the workers were clueless about what to do, and the fire department helped free him.
He managed to get stuck despite being supervised in a public restaurant playground area. And it was good he was being watched...I shudder to think of what happened if the play area had shifted just right that it could have crunched his arm.
There are certain risks to playing in a playground, and we accept that. But this was more of a maintenance issue. And it created a safety issue. I wanted to contact Burger King and let them know that maybe they might want to be careful about this...you know...inspect their playground equipment once in awhile.
Sin three: child-eating playground equipment.
Contacting Burger King was a challenge, to say the least. They had no Twitter account (Twitter launched in 2006, after all...); I could find no contact on their web page, there was no sign of an email address. The rest of the world had some form of electronic contact. Burger King gave customers an electronic middle finger.
Not kidding. A quick check on the Internet Archive at http://web.archive.org/web/20071006000851/http://burgerking.com/companyinfo/contactus.aspx (late 2007) came right out and said "E-mail communication is not accepted." In 2007. What the hell?
Sin four: about as tech savvy as a three year old, or perhaps K-Mart.
I did something that really just made me angry. I wrote them a letter. An actual, dead-tree, ink on paper letter. I mean, it was printed...I'm not a luddite and I know how to use a word processor. But it still irritated me that asking them (or their franchisees) to have some standard of not smooshing children in their playground equipment really shouldn't cost me money.
What I wanted was an acknowledgement that they'd do something to keep other kids from getting eaten by equipment. What I got was bupkis. I never heard back from Burger King. Not even some offhanded blame against their franchisee, pretending they have no control over how their image and name is represented. Nada. Zip.
My letter went into some great abyss of customer fuck-offs.
Sin five: at least have the decency to acknowledge something like this. There was a fire department called, you bastards.
What got me musing on the myriad reasons I avoid this BK when at all possible is my son's recent insistence...begging, really...that his special day out include a trip to Burger King for morning breakfast. I opened a web browser and it connected to my employer's website. I noticed that there was an oddly shaded and content-empty bar along the bottom of my web page.
Viewing the source showed some issues with loading something from an advertisement link. Which was strange, since my employer doesn't shove any weirdball ads like this at their users (I think it was blanked out because of an ad-block extension active in my browser at the time, rendering whatever their ad bar was supposed to be, blank, instead.)
I opened a new browser tab and navigated to my employer again, but this time used the https: link. That time the advertiser disappeared. See, using an encrypted point-to-point connection makes it difficult to inject extra HTML into my browsing session without me knowing...
...which meant Burger King, the company that for so many years makes it hard to actually contact them with any feedback and displayed a stunning lack of customer interaction and tech savviness, was injecting unwanted ads (and probably monitoring) my web browsing over their wifi.
Sin six: You're monitoring and injecting content into my browsing? Really?
I'm assuming that there was something about this buried in the legalese agreed upon when clicking to use their wifi, although to be honest, I don't remember clicking on anything in order to get on their wifi. And when it comes to public wifi, there's always the danger of your session being hijacked if you're not using a VPN or some other form of encryption. But still...another reason to dislike you?
I'm not sure how you stay in business other than being convenient for people in an area that doesn't have a lot of income. You're close to a retirement home, and getting to the Dunkin' Donuts would mean crossing the street in an area that has horrible crosswalk support. Maybe the Walmart model is at play; cheap food is cheap, so who cares about customer experience?
I even made a passive aggressive complaint on Twitter about their monitoring of web traffic and injecting code into the session (believe it or not, BK is on Twitter now...they joined in 2010. Yeah...little late to the party.) They said nothing. I've made irritated remarks about Dell and Time Warner and had them reply to me without even trying to talk to them. BK doesn't give a damn.
Sin seven: you monitor customer web traffic but not your mentions in Twitter. Get with the program.
Last I went to their website to contact them about the web-jacking. I clicked on the contact us page. It redirects to a "tellusaboutus.com" website.
They outsourced the "contact us."
A totally outside company handles "contact us" for Burger King.
Sin eight: ...I...no. That's enough.
Many companies are at least trying to not suck. They show signs of understanding how to engage with customers. They monitor for signs that customers want to communicate with them, want help, want feedback, want acknowledgement (hey, hear of Netflix? Or Dominoes? Or any of the dozens of other companies that will get in the news with some playful banter on Twitter with customers?)
Most companies at a minimum make it easy to email them with complaints or suggestions. Hell, I've had companies that pester me for feedback via email.
Burger King is like a digital brick wall. They seem to actively not want customer feedback.
...I suppose that kind of explains quite a bit.
I'll look forward to what the next five years holds for Burger King. If McD's is feeling the financial pinch, it may not be long before Burger King topples over. Can't say I'll miss them.
Tuesday, August 25, 2015
Haiku OS...Noticed Something a Little Strange
I was updating my Haiku VM and noticed something a little strange in Top. I checked some Ps output and...yup. It's there too.
Do you see the weirdness?
If not, I'll try making it more obvious.
The "Big Brother" one was the first thing that caught my eye. Was something infecting Haiku?
Who would bother targeting an operating system with a sliver of market share? An OS still in development, for that matter? Perhaps a pissed developer?
These thoughts were going through my head as I weighed the decision to shut down the VM in case it was doing something to probe the network.
I did a quick check online for more information about this...surely someone else had noticed weird processes like that. I found that someone had indeed noticed and asked about this awhile ago; they were pointed to the AppManager.cpp source in Github.
So..."Big Brother" is a humorous way of naming a watchdog process. What about the ancient movie reference? I found a chat transcript from someone else who was rather puzzled as well:
So they're...intentional?
I understand geek humor being integrated with different projects. I'm hardly surprised when I hear there's some little easter egg hidden in the source code, or when there's a somewhat obvious integration of technology with a geeky cultural icon (like OS/2 having a release named Warp.)
I'm not quite so sure about the wisdom of having something with ominous connotations being used as a thread name in a process list, especially in the age of Edward Snowden. And there's something about having a reference to Austin Powers that just...doesn't fit at all into the project. It's not in a theme. It's not...anything. An inside joke? Is it trying to date the project in some way?
At best these are quirks that make you pause and wonder if someone drank too much before making a commit. At worst doing things like this adds to a perception that the project isn't really meant to be taken seriously. At least, in my opinion.
I'm all for fun jokes and geeky humor. I love the little hidden easter eggs in out of the way places. I'm just not sure about humor that pops out in a nonsensical manner like a hernia bulge with no rhyme or reason, hinting that it's up to something nefarious.
Do you see the weirdness?
If not, I'll try making it more obvious.
The "Big Brother" one was the first thing that caught my eye. Was something infecting Haiku?
Who would bother targeting an operating system with a sliver of market share? An OS still in development, for that matter? Perhaps a pissed developer?
These thoughts were going through my head as I weighed the decision to shut down the VM in case it was doing something to probe the network.
I did a quick check online for more information about this...surely someone else had noticed weird processes like that. I found that someone had indeed noticed and asked about this awhile ago; they were pointed to the AppManager.cpp source in Github.
So..."Big Brother" is a humorous way of naming a watchdog process. What about the ancient movie reference? I found a chat transcript from someone else who was rather puzzled as well:
So they're...intentional?
I understand geek humor being integrated with different projects. I'm hardly surprised when I hear there's some little easter egg hidden in the source code, or when there's a somewhat obvious integration of technology with a geeky cultural icon (like OS/2 having a release named Warp.)
I'm not quite so sure about the wisdom of having something with ominous connotations being used as a thread name in a process list, especially in the age of Edward Snowden. And there's something about having a reference to Austin Powers that just...doesn't fit at all into the project. It's not in a theme. It's not...anything. An inside joke? Is it trying to date the project in some way?
At best these are quirks that make you pause and wonder if someone drank too much before making a commit. At worst doing things like this adds to a perception that the project isn't really meant to be taken seriously. At least, in my opinion.
I'm all for fun jokes and geeky humor. I love the little hidden easter eggs in out of the way places. I'm just not sure about humor that pops out in a nonsensical manner like a hernia bulge with no rhyme or reason, hinting that it's up to something nefarious.
Sunday, August 23, 2015
Upgrading Raspberry Pi From Golang 1.4x to 1.5
Some additional notes with my adventures working on an older ARM-based Pi computer and Go. Basically this is notes...not a how-to, but handy reference, so it will probably be a bit more terse than my usually over-narrative writing style.
First I logged in to the Pi and ran screen, because I'm remote on a free wifi connection 150 miles from the Pi and I know it's a hijacked connection from Burger King so trusting its reliability is iffy at best.
Second, I couldn't get the "git checkout go1.5" command to work. So I took my existing ~/go directory and moved it to another directory. Go is now self-bootstrapped...it likes the previous build being available. And it looked for a specific subdirectory. So...
mv ./go ./go1.4
git clone https://go.googlesource.com/go
cd go
git checkout go1.5
cd src
./all.bash
From there it went through the build process, compiling Go with Go (nifty, huh?)
First I logged in to the Pi and ran screen, because I'm remote on a free wifi connection 150 miles from the Pi and I know it's a hijacked connection from Burger King so trusting its reliability is iffy at best.
Second, I couldn't get the "git checkout go1.5" command to work. So I took my existing ~/go directory and moved it to another directory. Go is now self-bootstrapped...it likes the previous build being available. And it looked for a specific subdirectory. So...
mv ./go ./go1.4
git clone https://go.googlesource.com/go
cd go
git checkout go1.5
cd src
./all.bash
From there it went through the build process, compiling Go with Go (nifty, huh?)
Friday, August 21, 2015
GoLang and MSSQL Databases: An Example
(I'll insert the sources for custom_DB_functions.go and sqltest.go at the end of the post.)
(Addendum - I recently ran into an issue with the query of the database in the custom_DB_functions.go package where calls returned an invalid object. It looks like you have to call it with the schema as well [mydatabase.myschema.mytable] to get the call to work. See this StackOverflow question for details.)
I created a small utility recently that pulled data from a text file to display on a web page. I showed it to my manager, and he said we might be able to integrate it with a more useful bit of our infrastructure already in place, but in order to do that, it would have to talk to an MSSQL server for content instead of the text file.
I'd never really worked with testing that setup. In searching the Internet for examples, there are lots of fragments providing hints how to do it...but nothing really spelled out an example in tutorial form. So I kept notes and now I'm sharing what I learned for other beginners that want to experiment with integrating an MSSQL database with their Go application.
I'm only going to hit some highlights in the source; the source code has quite a bit of commenting and is pretty self-documented (but if you have questions...or suggestions/corrections...please leave a comment! In the blog comments, that is.)
Creating a Test Database Server
I wanted to set up an accessible test environment for anyone, not just people who happen to have a full-on SQL Server available. I fired up my Windows VM and downloaded MS SQL Server Express 2014. Free for most purposes!
I ran the installer (the 64 bit version with tools) keeping the defaults.
I had to enable network access to the database engine.
sqltest.go
(Addendum - I recently ran into an issue with the query of the database in the custom_DB_functions.go package where calls returned an invalid object. It looks like you have to call it with the schema as well [mydatabase.myschema.mytable] to get the call to work. See this StackOverflow question for details.)
I created a small utility recently that pulled data from a text file to display on a web page. I showed it to my manager, and he said we might be able to integrate it with a more useful bit of our infrastructure already in place, but in order to do that, it would have to talk to an MSSQL server for content instead of the text file.
I'd never really worked with testing that setup. In searching the Internet for examples, there are lots of fragments providing hints how to do it...but nothing really spelled out an example in tutorial form. So I kept notes and now I'm sharing what I learned for other beginners that want to experiment with integrating an MSSQL database with their Go application.
I'm only going to hit some highlights in the source; the source code has quite a bit of commenting and is pretty self-documented (but if you have questions...or suggestions/corrections...please leave a comment! In the blog comments, that is.)
Creating a Test Database Server
I wanted to set up an accessible test environment for anyone, not just people who happen to have a full-on SQL Server available. I fired up my Windows VM and downloaded MS SQL Server Express 2014. Free for most purposes!
I ran the installer (the 64 bit version with tools) keeping the defaults.
I had to enable network access to the database engine.
- Open the SQL Server 2014 Configuration Manager
- Click SQL Server Network Configuration in the left hand column
- Open Protocols for SQLEXPRESS
- Make sure TCP/IP is "enabled"
- Right click TCP/IP, click Properties
- Click the IP Addresses tab
- Check that IP2 is in the local subnet
- Check that TCP Dynamic Ports is blank in the IPALL section
- Check that the TCP Port is set to 1433
- Restart the SQL Server (SQLEXPRESS) service
At this point a quick NMap scan of the VM showed that port 1433 was open (when I used the -Pn flag.)
There's another setting to change coming up...
There's another setting to change coming up...
Prepare Your Go Project to Talk to The SQL Server
Go projects talking to a SQL server need two components; a driver, and a library that abstracts that driver from the programmer. The driver depends on what type of server you're talking to, but the abstraction layer is pretty standard.
Since we're talking to MSSQL, we'll use the go-mssqldb driver. From your Go workspace run:
go get github.com/denisenkom/go-mssqldb
I decided I was going to create a test application that imported a custom package of functions that specifically accessed the database; that way I could import the resulting package to my existing application and make necessary alterations at that point.
In the package I imported the driver with the line
_ "github.com/denisenkom/go-mssqldb"
WHAT IS THAT UNDERSCORE?
The problem is that I don't use the package identifier for anything in the program; when I tried to compile it, the compiler will error. What I really needed from the driver was the initialization method; the underscore will run the init() function but ignore everything else, and the compiler won't complain.
The abstraction part...the generic SQL Go functions to interact with the database...are imported with
"database/sql"
That import is in my package (custom_DB_functions.go and sqltest.go.) The driver is only imported in the package.
The test program was literally a "add functions as I go along and verify they work" thing. Add a function, code it in the package, recompile...repeat until I had most of the functions I wanted to work.
In order to make the test program flexible, I imported the flag package and created entries for connecting to the database from the command line. Import with
import (
"flag"
)
In main(), I added a series of flags using
// Flags
ptrVersion := flag.Bool("version", false, "Display program version")
ptrDeleteIt := flag.Bool("deletedb", false, "Delete the database")
ptrServer := flag.String("server", "localhost", "Server to connect to")
ptrUser := flag.String("username", "testuser", "Username for authenticating to database; if you use a backslash, it must be escaped or in quotes")
ptrPass := flag.String("password", "", "Password for database connection")
ptrDBName := flag.String("dbname", "test_db", "Database name")
flag.Parse()
You can probably divine the meaning from the syntax, but these are in the form of
<variable> := flag.<type>("flagname", default setting if not provided at command line, "Help explanation")
The variable created is a pointer, and flag.Parse() evaluates the flags at runtime. The default values are set to the second argument if they're not changed at the command line, so you can use them as variables without having to set them to something before referring to them.
The first function I wanted to test was the creation of a database handle. It seems that the behavior of Open relies on the underlying driver; it may, as the docs say, validate arguments without a connection to the database, so it may be necessary to actually do something with the handle before active connections are made. At any rate, here's the function in sqltest.go:
db, err := sql.Open("mssql", "server="+*ptrServer+";user id="+*ptrUser+";password="+*ptrPass)
if err != nil {
fmt.Println("From Open() attempt: " + err.Error())
}
defer db.Close()
Now db is a handle to the database. Because of the way database connections are handled, you don't want to keep opening and closing them. Just defer the Close() until the program exits and that way you won't get goofy pooling/caching issues.
From what I can tell of the documentation the db handle must be kept open for the length of time that you're using it (don't close it until the program exits.) This lets the driver manage a pool of connections; when you operate on the database you use a connection from the pool. Those connection(s) you'll want to close so the connection gets returned to the connection pool. In this program I deferred the close of the db handle because the scope of the test program should keep it open until sqltest ends.
Basically Open() needs the type of database (MSSQL), the server IP or DNS name, the username, and password. I should also note that the username, if you're using Windows auth, the backslash must be entered twice so it's escaped properly or the username must be encased in quotes. for example, the connection at the command line might look like:
From what I can tell of the documentation the db handle must be kept open for the length of time that you're using it (don't close it until the program exits.) This lets the driver manage a pool of connections; when you operate on the database you use a connection from the pool. Those connection(s) you'll want to close so the connection gets returned to the connection pool. In this program I deferred the close of the db handle because the scope of the test program should keep it open until sqltest ends.
Basically Open() needs the type of database (MSSQL), the server IP or DNS name, the username, and password. I should also note that the username, if you're using Windows auth, the backslash must be entered twice so it's escaped properly or the username must be encased in quotes. for example, the connection at the command line might look like:
./sqltest -password=HelloThere -server=192.168.254.222 -username=MySystem\\testuser
Notice there's two backslashes in the username?
Skipping ahead a little, sqltest.go makes a call to PingServer(). In the package, PingServer looks like:
func PingServer(db *sql.DB) string {
err := db.Ping()
if err != nil {
return ("From Ping() Attempt: " + err.Error())
}
return ("Database Ping Worked...")
}
Skipping ahead a little, sqltest.go makes a call to PingServer(). In the package, PingServer looks like:
func PingServer(db *sql.DB) string {
err := db.Ping()
if err != nil {
return ("From Ping() Attempt: " + err.Error())
}
return ("Database Ping Worked...")
}
It's a rather simple function, and all it does is run a call to db.Ping and returns a string with an error or an affirmation that it's working. This created a working connection to the database (as discussed before), and also tested the ability to pass the database handle to a package function.
Something else to note in sqltest.go is that I called the initial Open() using sql.Open(), while the call to PingServer was simply PingServer(). The trick to that is in the import statement.
import (
"database/sql"
"flag"
"fmt"
"os"
. "custom_DB_functions"
"strconv"
)
The import for "custom_DB_functions" is preceded by a period. That allows me to refer to the functions without the preceding library name; if I preceded "fmt" with a period I should be able to use lines like Println("I'm printing this to the console!") instead of fmt.Println("I'm printing this to the console!"). I don't know what happens if I had multiple functions with the same name in different packages imported with the period...I would think the compiler would insist on the proper namespace for referencing those specific cases, but I haven't tested it.
Skipping ahead a little more there's a spot where I create the database if it doesn't already exist:
// If it doesn't exist, let's create the base database
if !boolDBExist {
CreateDBAndTable(db, *ptrDBName)
fmt.Println("********************************")
}
In the library, the call looks like this:
func CreateDBAndTable(db *sql.DB, strDBName string) error {
// Create the database
_, err := db.Exec("CREATE DATABASE [" + strDBName + "]")
if err != nil {
return (err)
}
// Let's turn off AutoClose
_, err = db.Exec("ALTER DATABASE [" + strDBName + "] SET AUTO_CLOSE OFF;")
if err != nil {
return (err)
}
// Create the tables
_, err = db.Exec("USE " + strDBName + "; CREATE TABLE testtable (source nvarchar(100) NOT NULL, timestamp bigint NOT NULL, content nvarchar(4000) NOT NULL)")
if err != nil {
return (err)
}
return nil
}
It was here that I encountered an error from the database.
Next Database Configuration Change: "CREATE DATABASE permission denied in database 'master'"
The actual SQL calls aren't all that difficult if you already understand SQL (the big notes involve not constantly opening and closing the database handle, and to use Query() when getting information back to process and Exec() when the reply is more or less either "this worked" or "error!"
In the above calls, you can see that the function creates a database, alters the AUTO_CLOSE setting on the database so it doesn't throw goofball errors when trying later queries, and then creates a table with particular attributes.
In the above calls, you can see that the function creates a database, alters the AUTO_CLOSE setting on the database so it doesn't throw goofball errors when trying later queries, and then creates a table with particular attributes.
What I got back the first time was the CREATE DATABASE permission denied in database 'master' error. That required some more tinkering with the database engine.
- Open SQL Server 2014 Management Studio and connect to the test database
- Click on your SQL EXPRESS instance
- Expand the Security folder
- Expand the Logins folder
- I created a local user on my system for testing, so I right clicked on BUILTIN\Users and select Properties
- Click on "Server Roles" on the left side
- Select "dbcreator" from the list of roles; this should be enough access
Clicking okay and re-running the database/table creation calls should work. CreateDBAndTable() and DropDB() were created mostly for testing purposes so I could periodically work with a fresh database without having to futz with Management Studio or other interface (and of course I learned how to do these tasks programmatically.)
Any Other Notes?
Between heavy commenting and naming functions and variables in a (hopefully) mostly obvious manner I think that the code is semi-obvious; most of what I would explain in text here would seem redundant or obvious. If you have questions feel free to leave a blog comment!
The only thing that comes to mind from the design point of view is the use of returning an error from the library functions instead of dumping something to the console. This means the caller is responsible for the presentation of messages; the application can decide if the returned message goes to the console or redirected to a file or possibly ignored.
Here's the source code! Hope it is somewhat helpful to someone! (Probably it will be most useful to me as a reference while trying to tune what I've been working on...)
Source Code
custom_DB_functions.go
// Package custom_DB_functions contains functions customized to manipulate MSSQL databases/tables // for our application // // Version 0.15, 8-13-2015 package custom_DB_functions import ( "database/sql" _ "github.com/denisenkom/go-mssqldb" "strconv" ) // PingServer uses a passed database handle to check if the database server works func PingServer(db *sql.DB) string { err := db.Ping() if err != nil { return ("From Ping() Attempt: " + err.Error()) } return ("Database Ping Worked...") } // CheckDB checks if the database "strDBName" exists on the MSSQL database engine. func CheckDB(db *sql.DB, strDBName string) (bool, error) { // Does the database exist? result, err := db.Query("SELECT db_id('" + strDBName + "')") defer result.Close() if err != nil { return false, err } for result.Next() { var s sql.NullString err := result.Scan(&s) if err != nil { return false, err } // Check result if s.Valid { return true, nil } else { return false, nil } } // This return() should never be hit... return false, err } // CreateDBAndTable creates a new content database on the SQL Server along with // the necessary tables. Keep in mind the user credentials that opened the database // connection with sql.Open must have at least dbcreator rights to the database. The // table (testtable) will have columns source (nvarchar), timestamp (bigint), and // content (nvarchar). func CreateDBAndTable(db *sql.DB, strDBName string) error { // Create the database _, err := db.Exec("CREATE DATABASE [" + strDBName + "]") if err != nil { return (err) } // Let's turn off AutoClose _, err = db.Exec("ALTER DATABASE [" + strDBName + "] SET AUTO_CLOSE OFF;") if err != nil { return (err) } // Create the tables _, err = db.Exec("USE " + strDBName + "; CREATE TABLE testtable (source nvarchar(100) NOT NULL, timestamp bigint NOT NULL, content nvarchar(4000) NOT NULL)") if err != nil { return (err) } return nil } // DropDB deletes the database strDBName. func DropDB(db *sql.DB, strDBName string) error { // Drop the database _, err := db.Exec("DROP DATABASE [" + strDBName + "]") if err != nil { return err } return nil } // AddToContent adds new content to the database. func AddToContent(db *sql.DB, strDBName string, strSource string, int64Timestamp int64, strContent string) error { // Add a record entry _, err := db.Exec("USE " + strDBName + "; INSERT INTO testtable (source, timestamp, content) VALUES ('" + strSource + "','" + strconv.FormatInt(int64Timestamp, 10) + "','" + strContent + "');") if err != nil { return err } return nil } // RemoveFromContentBySource removes a record from the database with source strSource. The // int64 returned is a message indicating the number of rows affected. func RemoveFromContentBySource(db *sql.DB, strSource string) (int64, error) { // Remove entries containing the source... result, err := db.Exec("DELETE FROM testtable WHERE source=$1;", strSource) if err != nil { return 0, err } // What was the result? rowsAffected, _ := result.RowsAffected() return rowsAffected, nil } // Query the content in the database and return the source (string), timestamp (int64), and // content (string) as slices func GetContent(db *sql.DB) ([]string, []int64, []string, error) { var slcstrContent []string var slcint64Timestamp []int64 var slcstrSource []string // Run the query rows, err := db.Query("SELECT source, timestamp, content FROM testtable") if err != nil { return slcstrSource, slcint64Timestamp, slcstrContent, err } defer rows.Close() for rows.Next() { // Holding variables for the content in the columns var source, content string var timestamp int64 // Get the results of the query err := rows.Scan(&source, ×tamp, &content) if err != nil { return slcstrSource, slcint64Timestamp, slcstrContent, err } // Append them into the slices that will eventually be returned to the caller slcstrSource = append(slcstrSource, source) slcstrContent = append(slcstrContent, content) slcint64Timestamp = append(slcint64Timestamp, timestamp) } return slcstrSource, slcint64Timestamp, slcstrContent, nil }
package main // Notice in the import list there's one package prefaced by a ".", // which allows referencing functions in that package without naming the library in // the call (if using . "fmt", I can call Println as Println, not fmt.Println) import ( "database/sql" "flag" "fmt" "os" . "custom_DB_functions" "strconv" ) const strVERSION string = "0.18 compiled on 8/11/2015" // sqltest is a small application for demonstrating/testing/learning about SQL database connectivity from Go func main() { // Flags ptrVersion := flag.Bool("version", false, "Display program version") ptrDeleteIt := flag.Bool("deletedb", false, "Delete the database") ptrServer := flag.String("server", "localhost", "Server to connect to") ptrUser := flag.String("username", "testuser", "Username for authenticating to database; if you use a backslash, it must be escaped or in quotes") ptrPass := flag.String("password", "", "Password for database connection") ptrDBName := flag.String("dbname", "test_db", "Database name") flag.Parse() // Does the user just want the version of the application? if *ptrVersion == true { fmt.Println("Version " + strVERSION) os.Exit(0) } // Open connection to the database server; this doesn't verify anything until you // perform an operation (such as a ping). db, err := sql.Open("mssql", "server="+*ptrServer+";user id="+*ptrUser+";password="+*ptrPass) if err != nil { fmt.Println("From Open() attempt: " + err.Error()) } // When main() is done, this should close the connections defer db.Close() // Does the user want to delete the database? if *ptrDeleteIt == true { boolDBExist, err := CheckDB(db, *ptrDBName) if err != nil { fmt.Println("Error running CheckDB: " + err.Error()) os.Exit(1) } if boolDBExist { fmt.Println("(sqltest) Deleting database " + *ptrDBName + "...") DropDB(db, *ptrDBName) os.Exit(0) } else { // Database doesn't seem to exist... fmt.Println("(sqltest) Database " + *ptrDBName + " doesn't appear to exist...") os.Exit(1) } } // Let's start the tests... fmt.Println("********************************") // Is the database running? strResult := PingServer(db) fmt.Println("(sqltest) Ping of Server Result Was: " + strResult) fmt.Println("********************************") // Does the database exist? boolDBExist, err := CheckDB(db, *ptrDBName) if err != nil { fmt.Println("(sqltest) Error running CheckDB: " + err.Error()) os.Exit(1) } fmt.Println("(sqltest) Database Existence Check: " + strconv.FormatBool(boolDBExist)) fmt.Println("********************************") // If it doesn't exist, let's create the base database if !boolDBExist { CreateDBAndTable(db, *ptrDBName) fmt.Println("********************************") } // Enter a test record boolDBExist, err = CheckDB(db, *ptrDBName) if err != nil { fmt.Println("(sqltest) CheckDB() error: " + err.Error()) os.Exit(1) } if boolDBExist == true { err := AddToContent(db, *ptrDBName, "Bob", 1437506592, "Hello!") if err != nil { fmt.Println("(sqltest) Error adding line to content: " + err.Error()) os.Exit(1) } err = AddToContent(db, *ptrDBName, "user", 1437506648, "Now testing memory") if err != nil { fmt.Println("(sqltest) Error adding line to content: " + err.Error()) os.Exit(1) } err = AddToContent(db, *ptrDBName, "user", 1437503394, "test, text!") if err != nil { fmt.Println("(sqltest) Error adding line to content: " + err.Error()) os.Exit(1) } err = AddToContent(db, *ptrDBName, "Bob", 1437506592, "Hope this works!") if err != nil { fmt.Println("(sqltest) Error adding line to content: " + err.Error()) os.Exit(1) } } fmt.Println("(sqltest) Completed entering test records.") fmt.Println("********************************") fmt.Println("(sqltest) Deleting records from a particular source.") // Delete from a source int64Deleted, err := RemoveFromContentBySource(db, "user") if err != nil { fmt.Println("(sqltest) Error deleting records by source: " + err.Error()) os.Exit(1) } else { // How many records were removed? fmt.Println("Removed " + strconv.FormatInt(int64Deleted, 10) + " records") fmt.Println("********************************") } // Get the content slcstrSource, slcint64Timestamp, slcstrContent, err := GetContent(db) if err != nil { fmt.Println("(sqltest) Error getting content: " + err.Error()) } // Now read the contents for i := range slcstrContent { fmt.Println("Entry " + strconv.Itoa(i) + ": " + strconv.FormatInt(slcint64Timestamp[i], 10) + ", from " + slcstrSource[i] + ": " + slcstrContent[i]) } }
Subscribe to:
Posts (Atom)



