Introduction: ServDuino - Build Your Own Arduino Web Server
More by the author:
About: I like to make things that move, sense, calculate, compute, blink, and make noise. I like making things that create high voltages, electrical arcs, and can light fluorescent bulbs at a distance. I like to do …
The Arduino Ethernet Shield is a sweet toy!!! Everybody, from the beginner who got their Arduino yesterday and hasn't slept since, to the hard core pro who learned c as a second language, can learn how to use the Ethernet Shield and have lots of fun with it. Its that awesome! As soon as I earned enough to get one, it was mine, but it took a while for me to figure out how to get my site running and on the web, and I had some trouble with the software, but once I got started, I designed a whole web server, which included a traffic meter, LCD screen, and SD card storage!
This instructable is going to show you how you can make your own "ServDuino" web server, and get it on the web without some of the roadblocks that I, and many others have faced.
Some of this instructable is a work in progress - specifically the code on the programming step. Not all of the code was released as of the publishing of this instructable because I am always improving or writing new code for my projects. I am also seeking help from the instructables community in writing code, not because I am lazy, but because I know that I don't have the most efficient code, as I have been programming in c for only a few years, most of which is spent in the world of Arduino.
Don't forget to rate!
I entered this into the Microcontroller contest, so if you think its awesome, or it helps you off the ground with the ethernet shield, vote for it! :)
This instructable is going to show you how you can make your own "ServDuino" web server, and get it on the web without some of the roadblocks that I, and many others have faced.
Some of this instructable is a work in progress - specifically the code on the programming step. Not all of the code was released as of the publishing of this instructable because I am always improving or writing new code for my projects. I am also seeking help from the instructables community in writing code, not because I am lazy, but because I know that I don't have the most efficient code, as I have been programming in c for only a few years, most of which is spent in the world of Arduino.
Don't forget to rate!
I entered this into the Microcontroller contest, so if you think its awesome, or it helps you off the ground with the ethernet shield, vote for it! :)
Step 1: What You Will Need
For this project, you need only 2 things minimum to make it work. Everything else is extras that you can include if you want to.
The Required Parts are:
An Arduino
An Arduino Ethernet Shield - Note: If you have an older shield, you will not be able to use an SD card with it. If you don't have a shield, get the newest version!!!
Optional Parts:
A box to put it in - Recommended
A Servo
An LCD Screen
Some LEDs
An SD Card
Even More Optional Parts - These are for external controls and/or sensors:
A Second Arduino
Whatever sonsers and controllers you want
Either a Long Cable or a wireless Transciever (like an Xbee)
The Required Parts are:
An Arduino
An Arduino Ethernet Shield - Note: If you have an older shield, you will not be able to use an SD card with it. If you don't have a shield, get the newest version!!!
Optional Parts:
A box to put it in - Recommended
A Servo
An LCD Screen
Some LEDs
An SD Card
Even More Optional Parts - These are for external controls and/or sensors:
A Second Arduino
Whatever sonsers and controllers you want
Either a Long Cable or a wireless Transciever (like an Xbee)
Step 2: Getting Started on the Web
The first thing you will need to do is put the Ethernet shield on the arduino. If you need help, line up the pin names on the shield to those of the Arduino. Then plug it in to your computer and connect an ethernet cable from your router to the Arduino. Open up Arduino IDE and run the WebServer example under arduino-00xx -> libraries -> Ethernet.
Open up your favorite web browser and type http://192.168.1.177 . You should see something like the image below. If it does not work, DO NOT PANIC!!! You probably have an Internet Service Provider the blocks port 80, the standard http port.
To test this, we will try one of the following ports 8080 or 8081. Go to the WebServer sketch and find the line that reads: Server server(80); It is at the begining of the sketch. Change 80 to 8080 or 8081. Now upload. To get on your site, however you will need to type: http://192.168.1.177:8080 or http://192.168.1.177:8081 . If it works, Great now you can move on to the next step!!! A great place to look at for forwarding help is portforward.com .
Before you get excited and call up your friend to get on, we have a problem. The IP address you site has can only be accessed by a computer connected to your router. Nobody else can access it. How do we solve this??? We use port forwarding. This can be easy or hard, depending on how you and your router make it. You will need to access you router's settings page and search until you see a link or tab or whatever that is named port forwarding, or port forwarding/port triggering, or similar. Go to that page, and select a new http service. Type the IP of your webserver, and set the starting and ending ports to 8080 or 8081. Save changes and now people can access you website from outside your router. But you will need a new IP. Your router's IP. Go to www.whatismyip.com . No go to your IP at http://this.is.my.ip:8080 or http://this.is.my.ip:8081. With this and is and my and ip being the respective numbers of your address.
Now call your closest friends up and have them look at it. But wait!!! There is still a problem, remembering all of those numbers can be a pain! We are not very good at remembering strings of numbers, but we are good at remembering words and names.
To fix this you will probably want to get an actual domain name, like example.com. The easiest way to do this is to go to dyndns.com and sign up for a free account and choose a free domain name. These include yourname .dyndns-something .com or yourname .dyndns.some2or3letterending . Now set this domain to your router's ip. To go to your site, you will type http://yourname.dyndns.whatever:8080 or 8081, depending on your server port. The http:// is needed in Internet Explorer, but not Firefox for ports other than 80.
The last step you can take (optional) is to make a webhop so that you don't have to type the :808x onto the end. I have http://teslalingeweb.dyndns-web.com pointed to my router's ip, and set http://teslaling.dyndns-web.com as a webhop to http://teslalingeweb.dyndns-web.com:8081 . For websites on port 80, the http:// is not necessary.
Now that you have the site on the web, you can build the server, and design your site!!
Open up your favorite web browser and type http://192.168.1.177 . You should see something like the image below. If it does not work, DO NOT PANIC!!! You probably have an Internet Service Provider the blocks port 80, the standard http port.
To test this, we will try one of the following ports 8080 or 8081. Go to the WebServer sketch and find the line that reads: Server server(80); It is at the begining of the sketch. Change 80 to 8080 or 8081. Now upload. To get on your site, however you will need to type: http://192.168.1.177:8080 or http://192.168.1.177:8081 . If it works, Great now you can move on to the next step!!! A great place to look at for forwarding help is portforward.com .
Before you get excited and call up your friend to get on, we have a problem. The IP address you site has can only be accessed by a computer connected to your router. Nobody else can access it. How do we solve this??? We use port forwarding. This can be easy or hard, depending on how you and your router make it. You will need to access you router's settings page and search until you see a link or tab or whatever that is named port forwarding, or port forwarding/port triggering, or similar. Go to that page, and select a new http service. Type the IP of your webserver, and set the starting and ending ports to 8080 or 8081. Save changes and now people can access you website from outside your router. But you will need a new IP. Your router's IP. Go to www.whatismyip.com . No go to your IP at http://this.is.my.ip:8080 or http://this.is.my.ip:8081. With this and is and my and ip being the respective numbers of your address.
Now call your closest friends up and have them look at it. But wait!!! There is still a problem, remembering all of those numbers can be a pain! We are not very good at remembering strings of numbers, but we are good at remembering words and names.
To fix this you will probably want to get an actual domain name, like example.com. The easiest way to do this is to go to dyndns.com and sign up for a free account and choose a free domain name. These include yourname .dyndns-something .com or yourname .dyndns.some2or3letterending . Now set this domain to your router's ip. To go to your site, you will type http://yourname.dyndns.whatever:8080 or 8081, depending on your server port. The http:// is needed in Internet Explorer, but not Firefox for ports other than 80.
The last step you can take (optional) is to make a webhop so that you don't have to type the :808x onto the end. I have http://teslalingeweb.dyndns-web.com pointed to my router's ip, and set http://teslaling.dyndns-web.com as a webhop to http://teslalingeweb.dyndns-web.com:8081 . For websites on port 80, the http:// is not necessary.
Now that you have the site on the web, you can build the server, and design your site!!
Step 3: Building the Web Server
While I am going to show you my web server design, it is totally up to you how you would like to layout and design it, but here are some guidelines.
I would suggest putting the arduino in a corner, so that there is access to the power, usb, and ethernet port and the SD card slot.
I would also reccomend using a sturdy box, the one I used wasn't very sturdy, and now the weight of the servo is making it sag in the middle.
If you plan to attach an lcd screen, solder wires to it, and connect them to the ethernet shield before putting it in the box, 8 wires are a lot to connect inside of the box!!
I would suggest putting the arduino in a corner, so that there is access to the power, usb, and ethernet port and the SD card slot.
I would also reccomend using a sturdy box, the one I used wasn't very sturdy, and now the weight of the servo is making it sag in the middle.
If you plan to attach an lcd screen, solder wires to it, and connect them to the ethernet shield before putting it in the box, 8 wires are a lot to connect inside of the box!!
Step 4: Connecting Your Devices
As with everything else with this project, you have a choice in what other stuff you want to add to your ServDuino. I have a few suggestions and rules that you should follow to make this a successful build.
The Servo.
The servo is simple to connect, attach the red wire to +5, the black to gnd and the yellow (or third wire) to a pwm pin. On the main board the pwm pins include 3,5,6,9,10,11. Pins 10 and 11 CANNOT be used, however, because they are used for the SPI communication with the ethernet shield and SD card. I would reccomend not using pins 5 or 6 either if you are using a delay or millis() because there is a jitter that can be annoying if your ServDuino is next to the computer!
The LCD Screen.
A character LCD screen can be connected with as few as 6 data pins. Which is good because there are exactly 6 digital pins left over. Now before you comment and tell me that there is actually 9 pins, let me explain. We've used pins 13,12,11,10 and our choice of 9,6,5 or 3. Thisl leaves 9 pins. BUT, we need a pin for the SPI communication to the SD card, which is pin 4. Next, we must remember that the Arduino uses the UART to upload its sketches, so we don't want to use pins 0 or 1, just in case. Now how do we connect the LCD screen with only 6 pins, like this:
Vss is tied to ground.
Vcc is tied to +5 volts.
Vo can be tied to ground, or you can attach a potentiometer to it.
RS is tied to a data pin. Remember what pin!!! Or better yet, write it down!
R/W is tied to ground.
E is tied to a data pin. Write it down!
DB0-DB3 are not connected.
DB4-DB7 are tied to data pins. Write it down!!
Backlight pins can be tied to their respective voltage source.
Analog Sensors:
Analog Sensors are very straight forward, but on the newer ethernet shield, and maybe the older ones too, there is 10K ohm resistors tied to +5v on analog pins 0 and 1. Watch out!!
Xbee and Serial communications:
Well, I don't have very much to offer here, I don't have any Xbees...yet...but I would like to offer some suggestions. Anytime you need to upload a sketch, you will need to either switch out the xbee jumper or disconnect the serial wires (if you are not going wireless), so you may want to try putting a jumper select on the server box. If you don't you will have to rip it apart to upload a sketch. That's not fun, trust me.
Step 5: Designing Your Site
Here is where you have to start thinking about what you want your site to look like and do, so that you can choose how you want to store your site's information.
If you want to have a very basic site, maybe only one page, or you want to be able to control some led's or light switches or whatever your heart desires, you should go with a site saved in the Arduino's ram.
If you want to have similar capabilities, but you want to have more than 1 page, you will want to consider using the code with PROGMEM.
If you want a regular site with pictures and videos and lots of pages, or whatever, you will want to look into using an SD card.
Limitations and features:
With the ram option, the site can be only like 1500 characters (of html codes, not of actual text) which is very limited, but it is good for a simple led (the led could be anything actually) controller.
With the PROGMEM option, you are much less limited. With an atmega328, you can store up to 12,000-13,000 html characters!!! This is suited for home automation or a multipage site w/o an SD card, but you cannot store pictures or files on it, and you are still limited to 12k characters.
The SD card is the best choice for a large site, with lots of scripting, pictures, pages, etc. But as of now, it cannot be used to control leds over the web, but I'm working on it though! I am also working on directories and using the SD library instead of the sdfatlib (but don't expect it for a while because my main focus is on a led controller and directories, unless the 'ibles community would like to help!)
If you want to have a very basic site, maybe only one page, or you want to be able to control some led's or light switches or whatever your heart desires, you should go with a site saved in the Arduino's ram.
If you want to have similar capabilities, but you want to have more than 1 page, you will want to consider using the code with PROGMEM.
If you want a regular site with pictures and videos and lots of pages, or whatever, you will want to look into using an SD card.
Limitations and features:
With the ram option, the site can be only like 1500 characters (of html codes, not of actual text) which is very limited, but it is good for a simple led (the led could be anything actually) controller.
With the PROGMEM option, you are much less limited. With an atmega328, you can store up to 12,000-13,000 html characters!!! This is suited for home automation or a multipage site w/o an SD card, but you cannot store pictures or files on it, and you are still limited to 12k characters.
The SD card is the best choice for a large site, with lots of scripting, pictures, pages, etc. But as of now, it cannot be used to control leds over the web, but I'm working on it though! I am also working on directories and using the SD library instead of the sdfatlib (but don't expect it for a while because my main focus is on a led controller and directories, unless the 'ibles community would like to help!)
Step 6: Storage Options
This is a more in depth view of the storage options I explained in the previous step.
Ram:
The first option is to store the site in the Arduino's ram. This is the simplest way because you simply call the client.print() function and write your html code inside it. It is limited to about 1500 characters, because the Arduino only has 2048 bytes of ram.*
Ex: client.print("<html>...</html>");
Program Memory:
Wouldn't it be great if you could store the code in the program memory instead of the ram. Well, actually, the above option does that, but the compiler then makes the micro read all of that out into the ram upon start up. How do we fix that? We use the PROGMEM Library, pgmspace.h. This tells the compiler that we want to leave the data in the program memory. Now we are able to have 12k characters.* The only problem is that the micro tends to freeze up easily, but I have a workaround for that in the Going further step.
The SD Card:
When It comes to storage amount, the SD card is definitely the best choice. You can have up to 64GB of memory!! The biggest problem is that it is not recommended for the Duemilanove with ATMEGA168 or older, because it compiles at about 16000 bytes at minimum. I rarely have freezing problem with the SD Card, unlike with the PROGMEM code, and I am very close to getting a working led controller code.
Ram:
The first option is to store the site in the Arduino's ram. This is the simplest way because you simply call the client.print() function and write your html code inside it. It is limited to about 1500 characters, because the Arduino only has 2048 bytes of ram.*
Ex: client.print("<html>...</html>");
Program Memory:
Wouldn't it be great if you could store the code in the program memory instead of the ram. Well, actually, the above option does that, but the compiler then makes the micro read all of that out into the ram upon start up. How do we fix that? We use the PROGMEM Library, pgmspace.h. This tells the compiler that we want to leave the data in the program memory. Now we are able to have 12k characters.* The only problem is that the micro tends to freeze up easily, but I have a workaround for that in the Going further step.
The SD Card:
When It comes to storage amount, the SD card is definitely the best choice. You can have up to 64GB of memory!! The biggest problem is that it is not recommended for the Duemilanove with ATMEGA168 or older, because it compiles at about 16000 bytes at minimum. I rarely have freezing problem with the SD Card, unlike with the PROGMEM code, and I am very close to getting a working led controller code.
Step 7: Programming!!
Here is where the fun - or frustration - begins!! You are now ready to upload the sketch of your choice and test your web server!!!
I have a bunch of sketches here for download. I will be listing them in the following categories: RAM, PROGMEM, SD. I will give a quick rundown of each sketch and what its functions are. I will provide a sketch with each new function. It is up to you to combined them in whatever combination you want.
RAM:
RAMWebServer_Basic - Basic Ram WebServer Sketch.
RAMWebServer_Control - Code to control and LED included.
*More coming soon!!
PROGMEM:
*More Coming Soon!!
SD:
IMPORTANT: The SDFat Files will not compile if you don't have SDFatLib and/or you have the SD.h library (included in Arduino-0022)
SDFatWebServer_Normal - Basic SD Card WebServer Sketch
SDFatWebServer_Fast - Contains a buffer between SD card read and Client write functions, making uploading much faster
SDFatWebServer_Normal_404 - Has code for a customized 404-file not found page, called 404error.htm
SDFatWebServer_Fast_404 - Has code for the 404 page and the buffer
SDFatWebServer_Normal_LCD - Has code for an LCD screen hit counter
SDFatWebServer_Fast_LCD - Has code for the LCD and the buffer
SDFatWebServer_Normal_Servo - Support for a web traffic servo meter.
SDFatWebServer_Fast_Servo - Servos and Buffers, what better?
*More coming soon!!
I have a bunch of sketches here for download. I will be listing them in the following categories: RAM, PROGMEM, SD. I will give a quick rundown of each sketch and what its functions are. I will provide a sketch with each new function. It is up to you to combined them in whatever combination you want.
RAM:
RAMWebServer_Basic - Basic Ram WebServer Sketch.
RAMWebServer_Control - Code to control and LED included.
*More coming soon!!
PROGMEM:
*More Coming Soon!!
SD:
IMPORTANT: The SDFat Files will not compile if you don't have SDFatLib and/or you have the SD.h library (included in Arduino-0022)
SDFatWebServer_Normal - Basic SD Card WebServer Sketch
SDFatWebServer_Fast - Contains a buffer between SD card read and Client write functions, making uploading much faster
SDFatWebServer_Normal_404 - Has code for a customized 404-file not found page, called 404error.htm
SDFatWebServer_Fast_404 - Has code for the 404 page and the buffer
SDFatWebServer_Normal_LCD - Has code for an LCD screen hit counter
SDFatWebServer_Fast_LCD - Has code for the LCD and the buffer
SDFatWebServer_Normal_Servo - Support for a web traffic servo meter.
SDFatWebServer_Fast_Servo - Servos and Buffers, what better?
*More coming soon!!
Attachments
Step 8: Going Farther
If all went well, you now have a working ServDuino web server of your own!! Now you can try different codes, and even write your own, and if it is good, send it to me and I will post it on the instructable and credit you for it!!!
PROGMEM fix:
The server likes to freeze up a lot when running using the PROGMEM storage. An easy fix would be to use the watchdog timer, but the problem with that is that when it resets, the Arduino goes into an infinite loop, because there is something messed up in the bootloader. The easiest thing is too make an external watchdog with a 555 timer, but if you want to keep it software based, you will need the ladyada bootloader . This enables you to use the watchdog timer to reset the frozen Arduino. Now you have another problem with going software though. You still need to reset the ethernet shield. How? Cut the reset pin off of the shield then run a wire from it to a pin of your choice. Now you just set the pin high in setup() then bring it low for a few microseconds before you initialize the ethernet chip later on in setup(). A watchdog can also be useful in the RAM and SD options.
SD Card favicon.ico:
When I was testing the SD Card codes, I had the USB cable plugged into a computer and had a terminal monitoring the incoming serial. When somebody gets on the site, the Arduino sends what page they just requested. I noticed that whenever somebody went to the homepage, there was also a request for a favicon.ico. The favicon.ico is the little icon in the address bar, on the tab, and in the favorites folder next to the site. To get your own favicon.ico, you will need to find an online favicon generator, (google that) make your icon, then download it and put it on your sd card, in the root directory (not in any folders). If you have the site saved as a favorite, delete the favorite, and go to your site. You should see it appear. Note: If you go into paint and make one and save it as an .ico, internet explorer will reject your favicon. It will work in firefox and chrome, but not in ie. If you want you can also make the browser request the favicon by putting this into your homepage html file(index.htm): <link rel=shortcut icon type=image/x-icon href=favicon.ico>
Now go have fun, and rule the web!!
PROGMEM fix:
The server likes to freeze up a lot when running using the PROGMEM storage. An easy fix would be to use the watchdog timer, but the problem with that is that when it resets, the Arduino goes into an infinite loop, because there is something messed up in the bootloader. The easiest thing is too make an external watchdog with a 555 timer, but if you want to keep it software based, you will need the ladyada bootloader . This enables you to use the watchdog timer to reset the frozen Arduino. Now you have another problem with going software though. You still need to reset the ethernet shield. How? Cut the reset pin off of the shield then run a wire from it to a pin of your choice. Now you just set the pin high in setup() then bring it low for a few microseconds before you initialize the ethernet chip later on in setup(). A watchdog can also be useful in the RAM and SD options.
SD Card favicon.ico:
When I was testing the SD Card codes, I had the USB cable plugged into a computer and had a terminal monitoring the incoming serial. When somebody gets on the site, the Arduino sends what page they just requested. I noticed that whenever somebody went to the homepage, there was also a request for a favicon.ico. The favicon.ico is the little icon in the address bar, on the tab, and in the favorites folder next to the site. To get your own favicon.ico, you will need to find an online favicon generator, (google that) make your icon, then download it and put it on your sd card, in the root directory (not in any folders). If you have the site saved as a favorite, delete the favorite, and go to your site. You should see it appear. Note: If you go into paint and make one and save it as an .ico, internet explorer will reject your favicon. It will work in firefox and chrome, but not in ie. If you want you can also make the browser request the favicon by putting this into your homepage html file(index.htm): <link rel=shortcut icon type=image/x-icon href=favicon.ico>
Now go have fun, and rule the web!!