Home > September 07 Roundtable Part 2

Save

Edit

Cancel

More actions

September 07 Roundtable Part 2

Tags:

Church IT Roundtable at Granger, September 26, 2007, Part 2

[It sounded like it started in the middle of a conversation, someone is talking about Postini?]

Speaker

We started out with Postini, we were using an interval product from Quest software, I think, and it was taking a person almost full-time to deal with the spam. We moved to Postini and everybody did really well with it. Then MS Logic came in and said they could do it cheaper, so we went with them. It was ok but not as good as Postini. Then Postini called back a year later and said they would beat the MS Logic price so we went back with them. So you could definitely negotiate with them.

Speaker

What’s their policy on internal spam and distribution lists? Is that a problem?

We have a written policy that is part of our network user policy which is 14 page document that everybody is supposed to read. Basically it dialogue about not selling things, that kind of stuff. We don’t have an alternative at this point.

Yes, you have an alternative, it’s called bulletin boards! Seriously. You want [Time Stamp00:02:02] to sell something, put it on the bulletin board.

About a year and a half ago, we did a breakdown on how much it costs the church for every all-staff email that was sent out. It was about $26 per user for every all-staff email that went out. So he really put the clamp down. So very few all-staff emails are sent out now.

Our policy doesn’t have a good solution for that. Until we have a viable solution, we try to never say no, but we try to offer options.

I spend an hour with every new employee and a big part of that time is spent lecturing about what to send all-staff emails and what NOT to send all-staff emails. We put bulletin boards in every stair well and told everyone to use those for stuff for sale and things like that.

Jason

File storage is a big thing too. The big thing with sans is storage, so that your spam solution, we have a lot of redundancy. So [Time Stamp00:06:56] with the new storage systems coming out, the redundancy is taken care of.

There’s a new storage server. File level and block level. [Could not hear this speaker very well]

You cannot solve personnel problems with technology. And overuse of emails and that type of thing is a personnel issue, not a technology issue. We had that kind of thing going on, but we had a convenient time and opportunity shortly after that. I went to a person and said, “It looks like you don’t have enough to do, I have another job for you.” I did that twice and we don’t have that issue anymore. It’s a management issue.

And I guess that goes hand in hand. One other thing I learned a short time ago is let the leadership make some of those decision rather than the IT. We struggled [Time Stamp00:08:50] with the My Space issue, and we as an organization weren’t prepared to deal with all the pieces of the puzzle, so the leadership team made the decision. Because of the way we IT people are wired, we want to be the police, but we need to push that up to the leadership level, and that has been very helpful to me.

Jason

Context filtering was one of the topics, so let’s talk to that. Just what are people using?

We are still struggling with that, partly because they are expensive for high quality content filter, so we’re trying to figure out what the best one is because we have public wireless access as well as internal staff. We have separate policies on both of those and I’d like something that will separate that out by vlans. With our public Internet, I had it locked down, so you can’t just change your IP settings, you have to go through open DNS. [Time Stamp00:11:48] Our internal DNS control is also. But that’s not the best solution. It’s more of an content analysis tool, it can overblock a lot of things that it doesn’t recognize.

We’ve had really good luck with our Sonic Wall and it does so many other things for us to. We pay a user-subscription but that covers any virus, it covers content-filtering updates, all that stuff. We did a dedicated device a few years back, the I-prism [not sure exactly what he said] St. Bernard, it was great and flexible. It was relatively expensive. It’s the Pro 20-40, the content filter is just one of the functions it does for us. That’s our main firewall. Their support has been responsive.

Our biggest thing is, which I pushed back to the leadership, is if we have wifi, what’s our responsibility, and if a minor comes in with a laptop. We decided it is our responsibility because we provide Internet. I found that I was more restrictive [Time Stamp00:15:18] with my expectations of what we wouldn’t let our staff get to, but the leadership was expecting that we would give access to, part of that is a trust relationship with our staff and a good management structure and those things aren’t an issue because of that. But that had to be a leadership decision. So since we are using the same content-filtering for public and private, we were highly restrictive.

The problem we have right now is that we could do by-user filtering with the previous version of the Sonic wall software, but then every user would have to authentication every time they opened up a browser and that was not an option. Our leadership team quickly ruled that a no. So I would say push that to leadership and let them make that decision.

My concern is switching firewall is a much larger issue than just dropping a content-filter in place. Between us and the firewall, we’ve got ASA and we’re looking [Time Stamp00:17:38] at just getting the content-filtering appliance from Sonicwall not the full-blown.

We have Sonic wall at the very head of our network and we have an icebox behind it and part of what we’re looking at is moving at the SSLVPN [not sure about that] from Sonic wall. Part of our challenge right now is trying to figure out how we can continue to use our PC,,, right now you come in wherever you’re at, authenticate via our radius box and it looks like a struggle, and we’re willing to move away from the icebox but right now it’s a struggle to make that happen.

Jason

Content-filtering wise, we’re probably the least restrictive of all the churches I’ve talked to. We’ve had the same discussion on the Church IT podcast and again, from leadership down, when I first got here, we had no content-filtering, nothing. I enquired about that and leadership was like let’s not expect people to go where they should [Time Stamp00:19:07] go. But where we’re at now is that basically we block the worst of the worst and everything else is open, and we’re not watching any of that stuff. When I worked at the school, I spent half an hour every morning looking through the logs and sometimes some kid would figure out how to get to a Victoria Secret website…block! So I’m familiar with managing that stuff, I think it changes a lot if your church has a school involved with it, you really do need to be watching, or if there’s something in a public space.

How are you filtering for people who are on your staff?

Filtering has one problem, and that’s that creative people can always figure out a way to get around it. I hear every week at least of at least one person somewhere that gets fired because they get involved in the dark side of the net because there are so many ways around it. We believe very [Time Stamp00:21:13] strongly that you should have some kind of accountability software on every laptop and desktop so that whether you’re filtering or not, every website somebody visits is logged. And there are free solutions to that but our favorite is from Covenent Eyes. www.covenanteyes.com

What’s great about it is when you set people up with their Covenant Eyes account, and the licensing for churches is really good, like 10 users is $275 dollars a year, for 50 users, $1100 a year, not bad, but when you set them up, they set up as many as two or more accountability partners that will get an email every week showing every website they’ve been too and what’s good about that report is it scores the sites. Those that score unacceptable show up in the summary. You can also get a supervisory status if you are monitoring others. What’s great about this is people start avoiding the wrong sites, it’s a whole different mindset. It doesn’t do any filtering [Time Stamp00:22:56] and it doesn’t slow down any of your Internet access.

Along those lines, do you do different local client install on laptop.

We don’t. Tony mentioned a couple weeks ago that some staff were actually asking for something like that to be installed on their laptops because they didn’t want the stuff popping up while they were at home.

I’ve been testing Safe Eyes and it seems to work pretty good, it does a lot with reporting, you can check the reports online and you can see a list if you want or just say the ones that were bad.

I just read an article yesterday that 51% of Christian men in church leadership regularly visit porn sites. And 37% women. It was an interesting stat.

That’s why I still like filtering. I think that’s an obligation we have to our public and to our reputation. You have to protect.

Everybody has been talking about blocking sites and stuff, we do sort of like [Time Stamp00:25:10] Jason does, we get the worst of the worst, but we had to review this about 2 years ago and it was more of the approach that it wasn’t people going to bad sites, but it was people going to ESPN streaming during the day and doing bandwidth, now we have a new policy in place, there are certain sites, especially sports sites, that we block during certain hours, and also they get a little prompt that says, “Hey, make sure you’re going here for a business cause…”

How do you do that?

We use Websense, they have an option to say let people go to this site but give them a warning. It’s expensive.

Websense is an incredible product, we used it at the school where I was at but the cost is significantly steeper and is typically a little bit slower on the development side. Typically Wedsense is known for being behind the industry. That’s one red flag. But that might be why the product is rock solid.

Jason

[Time Stamp00:27:46] Let’s switch gears and go to the wireless.

You guys mentioned you have radius authentication, is that for sign-in and everything?

We use radius for our mail, we also use radius for our Rpc over actp so when you fire up Outlook, you come right through there. The radius stands between our iso box which is not a domain member.

We’re doing the same thing but on a domain-based site.

For the wireless, do you have two separate access points for public and private?

Up until this week, we’ve been using multiple ssip on everything so we vlan off our wireless by tagging it and vlan the public out. Staff uses the same access point but because their ssip is untagged. Right now we just added some access points.

What about, I know there are a couple of different security methods used over private, what sort are you using? [Time Stamp00:30:29] Shared, radius?

We’re using WPA, but out access points drop in how many people can connect to them because evidently it takes more over head to run WPA. We try to keep the key close to our chest.

We have an HP wireless network centrally managed and controlled. It essentially does the same thing. We actually have two private ssid’s right now, one is the same ssid and preshare key that we had, and I’ve also tested and got working a second ssid that uses radius, so you go in and use the Enterprise with MS Chat to authenticate to log in, so you can enable and disable any particular user’s access to the wireless by adding or removing them from the directory.

We’ve resisted going to radius because of the complexity.

It is complex. It took me a couple of days of experimenting to figure out how to get it functional and I’m still not comfortable

[three or four people talking [Time Stamp00:33:12] at once, something about certificates]

I’d like to get the certificates but I intentionally have increased password complexity, we previously had weak passwords, so I was making sure our users getting in externally had a much stronger password than everybody else.

Our big thing with the security of the wireless is if volunteers have to get on, even if you give out the web key, then you either have to keep changing, if somebody falls out of favor, you have to change everything.

If you’ve got volunteers, I was going to ask the obvious question about Mac authentication, how does that work? With radius?

We have some Macs and I have had trouble getting them to authenticate with radius but it is built into to set up and connect to a network with WPA Enterprise and WPA personal and you can enter using a password with the Mac directly. Sometimes it will lose the settings.

Has anybody solved that [Time Stamp00:35:56] losing the settings with Macs?

Maybe Tony, please.

I think VPN and WPA security are both very similar in capabilities. They can track users.

We have a weekly staff meeting, you get 100 people in the room, if everyone had to do VPN, you could totally crash it.

How many of your users are laptop vs desktop?

More and more, probably 50% if not more.

In our case, the budget numbers for what you get for new-hires hasn’t changed in the past 4 years because they kept the same number.

This is completely off the subject but it intrigues how do you as IT support that? Is the leadership aware that, oh it costs us $1500 bucks now but our cost is double or triple when it comes to our circulation of machines?

They are all about 3 or 4 years old, the hard part has been that we’ve grown so much, so it hasn’t hit fully yet. I’m telling them they need to [Time Stamp00:39:46] plan further out. But it doesn’t seem to register with them.

Jason

One of the other issues was monitoring wireless, deploying wireless, and setting up wireless.

For us, it’s been funny the evolution of our wifi, it went from throwing a radio up in the atrium area and if it’s on, it’s on, if it’s not, it’s not. Well then it became quickly that it needs to be on because now it’s a dependent solution, so for us we’ve now got the hardware but in a box, we’re deciding on the best place to put them.

The traditional route would be to get a wireless site survey, but those run $2000-$3000 to have a company come in. What we did last year is we replaced all of our core network stuff and built in the wireless HP and I ran into the same problem, we want to cover as much area as possible.

We purchased [Time Stamp00:43:08] Visi Wave [?], they have a 20% discount for non-profit if you ask.

If you think you might be interested in ever buying a fluke, contact them and they’ll send you out an omniview, which is $23,000, it may not be the one you want. They’ll send an engineer, but you can take that thing and plug the wifi in.

So Jason are you going with a targeted approach, like pinpoint sections that are wireless coverage?

Jason

We’d like to have full-coverage on the entire campus. That’s our goal.

Do you typically put them in hallways or do you find that out by site survey?

Jason

I walk around and say this is where we lucked out, we’ve got a volunteer that has one of those $5,000 software things so we put a few access points in and because our senior management team wanted wireless in the auditorium to be able to do wifi stuff on Saturday nights, so he took his laptop, walked all over the building, [Time Stamp00:45:33] we saw where the cold spots were, got some more access points in place. He’s been here for hours adding more access points. We were anticipating a lot more wireless people here this week than in the past.

We went to Willow to look at how they are doing wifi in their auditorium and it’s quite the set up, to get 7,000 wifi enabled, it’s a formula, they built a wifi structure. I think where we are at is that we want basic coverage in our conference rooms, but by next year, it’s going to be a different story.

Once you say public wifi is what we’re going to do. It’s going to be available for the Leadership Conference and then expected to be available for everything else after that for eternity. Once you establish something like that, it becomes a critical service.

We will advertise free wifi.

Sp
I sent out something to our coordinators, to be aware that if presenters need to be on the Internet, we need to be aware of that ahead of time so we can wire them in because Bluetooth, that’s the thing that really mucked the whole thing up, people use Bluetooth and the wireless goes really crazy.

Is anybody simulating any part of the wifi once they’ve done the site survey and get it out there, to say, well now that we’ve got this, this is what we can happen?

What Willow asked us to do in June was to simulate x-number load of your system and the catch was the Willow does the stream to us as a backup and our tech team has to have that backup, so that want to make sure we have [Time Stamp00:49:59] enough bandwidth coming into the building for the stream and to have enough bandwidth for the interactive experience happening with our attenders to facilitate that. Our question was how do you even simulate that? And how will your wifi interact and the load on your T1 coming in and all the other variable. I think you have to live it and let it happen. We sat down with our leadership and said here are the IT issues, all the pieces of puzzle. Budget related to personnel, service level agreement, those things. How do we implement so that the user experience is one that makes them want to come back and do it here again. Someone on your team has to have the interface. And then how about assessing how the IT is doing.

The struggle for me is not trying to be in on meetings that I don’t want me to be in and not creating the expectation that IT just wants to be in on everything, so what we’re trying to do currently [Time Stamp00:53:30] is I meet with my boss every week to dialogue about what happened in the leadership meeting the day before, so where I’m not physically at that table, I have a conduit for that. It’s difficult because there’s a balance between do you really need to be in those meetings. It’s not fun to be the person to have to come to the rescue at the last minute, especially when someone else knew a month ago that something needed to be done. I had to put my foot down, it is for the sake of making sure we all talk about it. I think Jeremy can attest that it’s easier to send an email, but it is not effective, go face to face. I have a monthly meeting with every leadership team member. That’s been awesome. It has given us a vehicle for complete open dialogue. We come with an agenda and have interaction. It’s been a year process.

I think it depends on the way your org chart is set up, there’s only so much we can do. We [Time Stamp00:56:57] are having an issue where we are doing as much as we can do to try to get in on those meetings, but people under people are doing things or driving things and people above them don’t know what they are doing, they don’t all know the priority. There is a whole hierarchy of authority that is messed up right now.

Establishing those relationships is what’s gonna do it. I don’t think you need to be in all those meetings, just make those relationships such that someone in the meeting can dialogue.

I think that’s a great point. It quenches the thirst of thinking you need to be in that meeting. You have that dialogue, but this approach takes care of that need, and it’s more productive for our team and our organization.

I challenge you to work on that. Interact more with how the departments converge. We should be reaching out more.

Jason

Consider, do you have the personality to be in those meetings? [Time Stamp01:00:23] some IT people just do more damage than good if they are in those meetings. So sometimes that interface between IT and Senior Management may not even be somebody in IT, it may be somebody in Communications that understand IT needs, it doesn’t have to be you, it just needs to be somebody that can interface for you.

That is key Jason, I’ve caused more arguments, it took me years to learn how to talk to that group of people, you can’t talk technology to them, they don’t get it. That’s why I have a job, I have to talk their language, and sometimes I have to talk up, sometimes down. I think that is key, find where you fit. Bring value and solutions to the table and you will get recognized. They will include you in sub-discussions and side conversations more.

Obviously, our solution doesn’t apply some places because it didn’t apply before I had Jeremy and another staff member aboard. Now that [Time Stamp01:02:20] I have four people, I’m a little bit removed from the day to day because Jeremy is the hero, so I am now able to start building those other relationships, our organization has said that’s my number one job priority as the IT Director. Part of that is clarifying with your leadership what they want you to do, and if I’m not doing to be that interface, is my supervisor going to be? If so, how do I educate my supervisor in a way that they can come to the table to speak intelligently about IT. They have to understand all the acronyms and how everything works, but allow them and equip them to go to the table and speak intelligently to their peers in a way that their peers understand and then be able to accurately represent the situation. It’s all about relationship building, which isn’t natural for me, but it’s better when I do it that way.

Being one of those people who goes into those meeting [Time Stamp01:04:31] and referring to my IT person, a lot of times the management team does not look at the five-year plan, they are looking at “does it work” so you need to communicate it for them, what is the bottom line? What is cost? Is it easy? And why? Give it without the jargon, in simplest terms. I think it is important for the IT manager to stress and emphasize why it is important to have a long-term plan. You’re information is valuable, it’s all about how you present it.

So in your position, what do you like to hear that justifies the cost that you don’t see a benefit to, like does it point, click and work, and we say, we can do it for this price for the short-term fix or here’s the way it should be done for long-term? What’s your take on that?

That’s basically what it is, you show me numbers or effectiveness, what it will cost now, what it will cost later. The best thing to tell somebody in management [Time Stamp01:06:52] is “we are a growing church” “we won’t be the same tomorrow as we are today” and then explain why the long-term is better.

One of the great reminders that our Executive Pastor reminded me of a couple weeks ago is IT is an industry, it facilitates industry to happen, it’s hard to keep that perspective. We can build in 52 layers of redundancy and backups and offsites, but as a ministry how do we need to function? We came to the conclusion that we can be down for 7 days as long as we can get Dean over here to bring a virtual server and our ACS database online within 4. If the building is gone because of a tornado or whatever, there are bigger needs than having the systems back online. Everything else that has to do with people, for me that was eye-opening. Really IT is not ministry, ministry is people, so we have to always come to the table when we are budgeting and realize how we can [Time Stamp01:08:40] come along ministry to excel and to equip them, so everytime I’m budgeting, it should driven out of the need for ministry rather than we need the greatest technology or whatever. And the helps the leadership team come to the conclusion that it is needs-based related to ministry needs and people rather than the latest greatest toy. We need that reminder frequently, that the church would survive if we didn’t have an IT department. Sure it helps us do things better.

My second week on the job at Northwood, we had a complete systems failure, we were down without email for a complete 7 days, server corrupted, welcome to Northwood. They felt that, they tolerated and survived.

They are getting so dependent on technology, the people standing there can’t do anything if they can’t get to their stuff.

And we have to look at it as are [Time Stamp01:11:27] we IT not doing the service and help them see the road they are going down. Educate the users, show them the potential risks, allow the leadership to say if that risk matters.

But this is where you have to have those relationships in place before the decisions, so you can talk about the risks, etc before the incident. They have to trust you enough.

[I couldn’t understand the name he said] has two great books about building a team and that it is pointless to have a team if you don’t trust each other. If I don’t have that trust relationship, they are not going to pay attention to anything I say.

Jason

I’ve heard a number of churches, people making changes Friday and then IT leaves for the weekend and weekend services and somebody has made a group policy change and this is how dissension occurs between technical arts and IT all the [Time Stamp01:13:58] time. Churches call me saying our technical arts and our IT people don’t get along, how do we get them to talk to each other. Part of it is not to make a global policy change on a Friday afternoon without having tested it and set up.

It goes back to the relationship, an example for us, I didn’t realize our crews got there so early on Saturday, I’m thinking we have a 6:00 service, they are probably not there until 3:00 or 4:00, so I can take the system down at 10:00 and do system maintenance and bring it back up by noon, that shouldn’t be a problem. Well my phone was ringing off the hook, they couldn’t believe I took it down at that time. I didn’t value them as a team, I didn’t realize their timeline or what their needs were, so now we don’t do anything until Monday, and they can trust to know that we are not going to pull the rug out from under them.

Jason

They trust that the system [Time Stamp01:15:27] is going to work.

We send on email out 4 or 5 days ahead of anytime we know that things are going to be down or that things are going to be changed, letting them know all the possibilities and limitations if we can. People are doing stuff 24/7, it’s hard to work around.

How do you as a communicator keep your communication noise level low enough that your users pay attention? I think I know my leadership team reads the alerts, but how do you keep your communications in a healthy range that one – you are keeping users informed but also not too much information, so they don’t immediately hit delete if they see an email from me?

I worked at Notre Dame before I came here in their IT department and they got to that point where they sent so many things, nobody read them. A constant blow, no one knew anything that was coming. When I came here, the email went down [Time Stamp01:18:44] and I got it back up, I send an email telling everyone what happened, everything is good now. Unless it is something that is huge, we don’t send all-staff emails.

Also, have a specified time every week for maintenance. We have Tuesday night from 5:00 on, everyone knows it will be down, plan on it, every week. Typically things work even when we are working on system center or building or whatever, we’re not usually impacting the network, but they still know that Tuesday night is maintenance night.

When we started the Tuesday night window, basically nothing happened on Tuesdays, now there are a lot of things happening, but they still know and most of them don’t need much that night.

We are not at the point where Jason and Ed are, we just started two weeks ago every first and third Monday night, it works for us, most of our staff is not even here on Monday, so we are [Time Stamp01:21:43] easing into it. It also orientates an opportunity for my volunteers, they know they can come during that time.

Two quick comments. Once you get monitoring systems in place, you will know when you can take things down. We monitor everything we do so I know when I can take something down. You can target the valley of usage. As far as notifying the users, we have developed just a simple little marquee application that runs across the top of our intranet instead of firing emails that they are not going to read, it says date, time, and short info, any questions call us.

Jason

Time to go eat. Meet back here at 1:00.

0 Comments Show recent to old