In January, one of my posts dealt with setting yearly goals for the legal department (click here to read).  One of the goals was sending out a client satisfaction survey, i.e., asking those in the business to weigh in on questions related to their satisfaction with the services provided by Legal.  There are a number of reasons why you would want to send out such a survey, primarily to gather helpful data and comments about how the legal department is performing and how it is perceived throughout the organization; the second being to use the survey as a tool to market and promote Legal by reminding people that the department is there, that it provides valuable services to the company, and that you are directly seeking their input into improving the quality and value of the services the legal department provides.

This edition of “Ten Things” will walk you through the process of creating, distributing, and analyzing the results of a client satisfaction survey.  Though the discussion deals with in-house counsel sending a survey to their internal business clients, the “why” and “how” apply equally to the relationship between outside/in-house counsel and outside attorneys can easily use this post to create a satisfaction survey to distribute to their in-house counsel clients.  If you intend to put out a survey this calendar year, now is the time to get started creating it for distribution after the summer holidays.

1. What is the purpose of your survey? The first thing you need to do is spend some time thinking about what it is you want to know or capture with your survey.  Typically, you will want the survey to give you insight into some measurable metrics showing how efficient or effective the department is, such as timeliness, quality of advice, accessibility of the lawyers, etc.  Likewise, you will probably want to get some general feedback on the department as a whole or about individual performers.  In addition to your own thoughts, gather a cross-section of your team and get their input on the survey as well.  Spending time upfront thinking about what you want to accomplish gives you an excellent road map to prepare the survey questions.
2. What tools can I use to create my survey? The easiest and best way to go is to use an online survey tool. This will allow you to create a professional-looking survey, let your clients complete the survey online, and provide you with the ability to tabulate and “slice and dice” the results in a number of interesting ways.  Check first to see if your company has its own internal survey generator.  If it does, use that tool as it will give you the widest variety of options/analytics (and you’ll probably have access to help and support in creating and managing your survey).  If not, there are a number of tools available online, typically a free version (which usually provides all the tools you need) or a paid version (which gives you more questions and more powerful analytics/flexibility).  Whether you use the free or paid versions will depend on the purpose of your survey, the number of questions, and the level of information and detail you need.  Three easy-to-use survey tools for you to consider are:  Survey Monkey, Zoomerang, and Survey Planet.  If you want to see more options, click here.  If you have the budget, you can also hire a professional consulting company to create a survey for you (e.g., Altman Weil).  But, unless you believe you have some really complex issues to sort out, I am confident you can create a valuable and useful survey on your own using the tools and ideas set out here.
3. Creating the questions. Once you have selected your survey tool, it’s time to prepare the questions.  To start, don’t make the survey too long or complicated.  If you do that, people will not complete it.  The maximum number of questions should be 20 or less.  You will also want to mix up the “type” of questions you ask, i.e., yes/no, rate on a scale, multiple-choice, open-ended, etc.  Mixing up the types of questions keeps the survey interesting.  Most of the online tools give you numerous templates to use to design your questions.  Your survey should not take more than 15-20 minutes to complete.  Additionally, before finalizing your questions, seek the input of your team.  Let them propose questions or give feedback on your questions.  Likewise, and more importantly, identify a group of clients and get their feedback on the types of questions they would like to be asked as well as on some of the questions you propose to use.  This will help you draft questions that clients will want to answer. You should think about the order of the questions so your survey has a logical flow and do your best to not build bias into the questions (another reason to have several folks review your questions).  Instead of listing a bunch of potential questions for you to consider, I am providing links to sample surveys that you can use as guideposts in creating your own questions (or copy verbatim if you wish).  I am attaching a survey I recently created using Survey Monkey (click here).  I used very similar questions in the surveys we circulated when I was general counsel at Sabre and Travelocity.  Here are links to other sample surveys, one from the ACC (click here) and one from eHarmony (click here).  These will give you a selection of potential questions and different question formats you can use.  If you want more ideas, type “law department client satisfaction survey” in your search engine and you will get other examples.
4. Confidentiality of responses. As you prepare your survey, you need to create a short introduction.  You will see an example in the sample survey I created.  Let the user know why you are seeking their feedback, why it is important that they participate, how long the survey will take to complete (e.g., “15 minutes or so”), and, most importantly, to let them know that their answers and feedback will be kept confidential and their identity will not be revealed or tied to any of the responses.  Confidentiality is important so as to obtain maximum participation and to generate more honest responses.  Still, one of the frustrating problems with an anonymous survey is that you have no ability to follow up with someone if they have particularly good or important feedback or ideas for the department.  One way around this is to add a place near the end of the survey to allow users to give their name and email address/phone number if they are willing to disclose that information.  You will see such a question at the end of my sample survey.
5. Who gets the survey/How often do you survey?  One critical question to answer before launching your survey is “who gets it”? This is important because you certainly want the survey to go to people who are frequent users of the department, but you do not want the survey only to go to “fans” of Legal, as the results will be skewed and will not be useful.  The first few surveys I sent out as general counsel were limited to vice presidents and above, along with some additional select employees.  While this gave us a pretty useful sample, we realized that a large group of people used the legal department regularly but were left out.  We then increased the survey distribution to include two levels down from vice president.  We got better results by doing this but it still seemed wrong to leave out anyone who wanted to take the survey.  As a result, we began sending the survey to every employee in the company, around 10,000 people.  We did not really add a significant amount of information by doing this (as many employees had very little, if any, interaction with the legal department), but I know the employee base felt better about the department because we included them and asked for their opinion – even if they did not complete the survey.  That goodwill alone was valuable to me.  Depending on your situation, my advice would be to just make sure you have a wide enough sample of people who will likely give both positive and negative feedback to the department– making for a much more useful survey. 

Another question is how often should you send out a survey (and when should it go out to your clients).   Any more than once a year is too much in my opinion and you will see response levels suffer from “survey fatigue” if you seek input more frequently than that  (as employees get bombarded with company surveys all the time).  Our surveys tended to go out about every 14 months or so.  I found that a little bit longer than one year between surveys got the best overall response rates.  As to when to send out the survey, be sure to avoid the summer (vacations) and the major holidays (e.g., in the U.S. avoid November and December and March – spring break).  In my experience, the best months are September or October because then you can use the results to plan for next year.  Also, coordinate with other departments to make sure your survey does not clash with or overlap another major survey going out to employees.  Keep the survey open for 30 days post-launch and send out several email reminders during the time that the survey is open, noting that you greatly appreciate people taking 15 minutes to complete it.  You can also promote the survey on your company’s internal intranet website, the legal department web site, company newsletter, at town hall meetings, on TV monitors around the facility, etc.  Ask your team to encourage their internal clients and colleagues to complete the survey.

6. Weighting the responses.  Not all responses are created equal.  In my experience, the opinions and thoughts of the frequent users of the department and those of the senior management tend to carry more weight.  The former because their frequent use means the feedback is generally more specific and useful.  The latter because they control the purse strings and their perceptions of the legal department as an important and valuable part of the company (vs. a black hole cost center) are critical as to whether the legal department thrives and has a seat at the table or is constantly fighting for funding and a voice.  One way to account for this is to weight the responses from those who are frequent users or who are part of senior management a bit higher than other responses.  For example, give a 1.25 weight to the former vs. a 1.00 weight to the latter.  To do this, you will need to ensure that you have the users identify themselves (generically) as “frequent” users of Legal — however you want to define that — and/or as senior management (e.g., VP and above, or SVP and above).  You will see an example of this on the first page or two of my sample survey.
7. Create a “word cloud.” You’ve probably seen a number of word clouds on a day-to-day basis.  These are charts that show different words in different sizes, shapes, and colors.  Using the available data set, the words that appear most frequently are “larger” and “brighter” than the words used infrequently.  In my sample survey, you will see a question that asks the user to list three words that they think best describe the legal department.  I used this same question in my actual surveys, created a database of the words provided by the users, and then used a word cloud generator to create a word cloud.  I included the word cloud chart in the survey results I shared with my team and with my CEO.  It is a great way to visually capture a snapshot in time as to what your clients think of the department.  You can see a list of word cloud generators by clicking here.   I used Wordle to generate my word cloud.  Here is the word cloud generated by my last survey:

Needless to say, we were pleased that “Responsive,” “Helpful,” and “Knowledgeable” were the three biggest, brightest words.  If you look closely, you’ll see that a couple of the words are not flattering. And that’s okay.  Just part of the deal and something we could focus on trying to make better (but we were glad these were among the “smallest” and “darkest” words in the cloud).

8. Analyzing the results.  Once your survey closes, it’s time to analyze the results.  A few things to understand about surveys.  First, don’t be disappointed if you do not get a “90%” response rate.  That would be amazing.  You should more realistically expect anywhere from 5%-10% of the people to respond (more if the survey pool is small).  Second, you will tend to get more responses from people who are “unhappy” with legal.  This is just human nature.  Don’t be put off by this.  Accept the criticism as just something you and the team need to work on to fix.  The fact that you asked for feedback has probably made some positive in-roads with these unhappy clients.  There will always be some people that no matter what you do, or how unreasonable their complaints, you will never satisfy — just a cold hard fact of in-house life.  Hopefully, none of those people are part of senior management.  If they are, that is a topic for another column down the road!  Depending on the capabilities of your survey tool, you should be able to slice and dice the data in a number of different ways, e.g., by region, by line of business, by type of legal matter, etc.  This will give you a lot of insight into where things are going smoothly or where you need to put in some work.  If you included the ability for people to write open-ended comments there is a lot of great information you can extract.   Look for patterns involving certain types of matters or certain groups or members of your team (good or bad).  All of these are areas for improvement or enhancement.  Prepare yourself for the odd curveball, e.g., someone complaining through the survey about things that are not part of Legal’s purview, such as procurement or IT system security.  The simple fact is that not everyone has a true understanding of what fits under Legal and what falls outside of Legal.   Your word cloud will give you additional insights as well.  Here is an example from one of my actual surveys showing the results regarding overall satisfaction with Legal year-over-year.  Given (like many in-house departments) we had budget and staff cuts leading into 2014, we were very pleased that 97% of our clients had the same or higher level of satisfaction with Legal vs. prior years:

 

9. Sharing the results. After you have been through the results yourself, it’s time to share them.  Most importantly, you will want to share them with your team.  I would first share the results with my department leaders and after they had a chance to digest everything, make the results available to everyone else.  We would then go over the results in detail as a team at an offsite or during one of our monthly staff meetings (click here re my column on running effective staff meetings).  Be sensitive about sharing the results with the full team if there are any extremely negative or unfair comments about any individual person set out in the comment sections.  If this happened, I would edit out names of those people from any version I shared broadly (though either I or their manager would show them the comments and discuss the matter).   After your team, you want to share the results with your CEO and senior management.  When you do this, your cover email should list the top three positives and the top three “things to improve” gleaned from the results. Be sure to note that you have plans in place or are making plans to address any particular shortcomings.  Finally, share some of the key results with the company at large.  You solicited feedback from a large group of people, let them see how the department fared (along with some general comments from you thanking them for their feedback and that you are putting into place action plans to work on issues identified through the survey).  You can post this on the legal department web page or work with corporate communications for the right way to proceed.
10. Create an action plan. Lastly, you want to be sure to use the data and the comments to make changes.  Develop a formal action plan to deal with the most important areas.  You do not need to do this alone.  Form a team with your department heads (and potentially others) and ask them to help develop plans for the department, themselves, and their teams.  You should also solicit ideas generally from the entire department so they feel vested in the process.  Ask everyone on your team (and don’t forget yourself) to make the relevant sections of the action plan part of their development plans for next year.   If users identified themselves for follow up, be sure to follow up with them and get their thoughts and ideas, and to show them that their feedback is important and valued.  Finally, share your action plan with the CEO (especially during any mid-year or full-year review) and senior management to let them know you are moving forward with fixing any problems.  Consider asking for a short meeting with your senior management peers to individually go over the key survey results and get their feedback on the action plan and/or the department in general.  Even if they don’t take you up on the invitation, they will greatly appreciate being asked for their input.

Client satisfaction surveys are great tools for in-house counsel.  You can easily create a useful survey in an hour or less and you will get a large amount of useful and actionable information from your clients.  The information will help you improve the delivery of legal services to the company and to build the brand of the legal department as a group that is constantly looking to improve, cares about what its clients think and need, is proactive and strategic, and is using cutting edge technology (i.e., the online survey) to build a better department.  However, don’t rely solely on surveys to get feedback.  Get out around the office and talk to your clients regularly.  There is still no substitute for a face-to-face conversation.

Sterling Miller

July 28, 2015

PS: For fun, I have created a short (5 minute) survey for you.  If you’d like to complete it, click here.  Follow me on Twitter @10ThingsLegal where I post articles and stories of interest to in-house counsel daily.

(If you find this blog useful, please click “follow” in the top right and pass it along to colleagues or friends and/or “Tweet” it. “Ten Things” is not legal advice or legal opinion.  It is intended to provide practical tips and references to the busy in-house practitioner and other readers. You can find this blog and all past posts at www.TenThings.net.)