Web Resources Depot, along with Web Appers is one of the sites I visit regularly to find useful web development tools and resources. You never know when a tool you come across will turn out to be useful in the future.
Tag: Software Development
Web Appers: One of my go to sites for web development resources
Web Appers is one of the sites I visit regularly to learn about new web development tools and resources.
WebAppers is a blog dedicated to share top quality open source resources for web developer and web designer daily. As a web designer, you’ll find
some of the best free icons, stock photos, brushes, fonts and design inspirations. As a web developer, you’ll also find some of the best Javascript
and Ajax components like modal windows, menus, galleries, tooltips, charts, calendars plugins and a lot more …
Email to my wife’s cousin: Software development resources
One of my wife’s cousins is graduating from college in the spring, and she was offered a job doing software development. She called and we spoke about what it’s like to be a software developer day-to-day. When we were done talking she asked me if I had any resources I could share to help her get started with learning about Java development. Here’s what I sent her.
Hey Cuz!
Below are three books that might be of interest to you. I don’t expect you to read all three. Most developers are lucky to read one book a year. These are just suggestions of books you might want to know about. I’d start with Head First Java.
Also, I have listed several concepts you might want to be familiar with. DON’T FREAK OUT! I know it’s a lot of information. I don’t expect you to learn all of that stuff. Just skim it so that if someone says, “we deploy our application on Tomcat,” you will have seen the word “Tomcat” before.
I recommend that you download a copy of the Eclipse IDE, install it, and use it to work through some examples from Head First Java.
Feel free to reach out to me if you have any more questions. You can reach me at this email address or on my cell phone at [REDACTED SON!].
Good luck!
Books
Head First Java, 2nd Edition by Kathy Sierra
If you want to learn basic Java programming, this is where I would start.
Effective Java (2nd Edition) by Joshua Bloch
This is my favorite Java book. I refer to it regularly. It’s the only one I have read cover to cover.
Code Complete: A Practical Handbook of Software Construction, 2nd Edition by Steve McConnell
This book is more about programming in general. Pick up a copy from your library if you can and thumb through it. It might have some interesting stuff in it.
Useful Website
Stack Overflow: http://stackoverflow.com/
This is the largest question and answer site for programmers. It is an invaluable resource if you have programming questions. Here’s a link to my Stack Overflow profile page if you want to see the kinds of questions a web developer might ask.
Concepts
Again, DON’T FREAK OUT! This is just a list of stuff an enterprise Java developer might work with. Just skim some of the web pages and familiarize yourself with some of the concepts if you have time.
- Open Source Software
- The Apache Foundation – They provide a lot of open source software.
- Java Build and dependency management tools
- Java Virtual Machine (JVM) – Every bit of Java code runs on top of a JVM. It’s important.
- Web Servers
- Apache HTTPD – This web server powers most sites on the web.
- NGINX – This web server is great at handling lots of traffic. Most large websites such as Youtube and Facebook make use of it.
- Lighttpd – Similar to NGINX in architechture.
- Servlet Container/Web Container/Application Server
- Apache Tomcat – One of the oldest and most popular web containers.
- IDE (Integrated Development Environment) – similar to Microsoft Visual Studio
- Text Editor – Every software developer will need to use a text editor regularly.
- Vi – This text editor is installed on every UNIX server everywhere. If you find out that you will be working on a UNIX based
platform such as Linux or BSD, these links will come in handy. (Vi reference). (Cute game meant to teach VI commands). - UltraEdit – The text editor I use. It is not free
- Sublime Edit – Another popular non-free editor
- TextPad
- Notepad++ – This one is free.
- Edit+ – Another popular one.
- Vi – This text editor is installed on every UNIX server everywhere. If you find out that you will be working on a UNIX based
- Unit Testing
- Rich Internet Application – A company light Delta probably has a few of these being maintained by internal development teams.
- Revision Control – These applications are used to keep track of changes to software. They are invaluable tools.
StackOverflow: An invaluable tool in my developer toolbelt
It feels odd to even mention this site since I use it so often, but I still meet software developers who have never heard of Stack Overflow.
“Stack Overflow is a question and answer site for professional and enthusiast programmers.” I’ve asked and answered a number of questions on the site, and it’s my go to place on the web when I have a software development or programming question of any kind. When I see Stack Overflow in my search results, it’s usually the link I click on first.
Here’s a link to my profile if you want to see the kinds of questions you can ask and get answers to.
gevent: coroutine-based Python networking library
I learned about this from the FLOSS Weekly podcast ᔥ. I’ve become more interested in event-driven frameworks lately because of Node.js and Vert.x.
From the gevent website:
gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libev event loop.
Features include:
- Fast event loop based on libev (epoll on Linux, kqueue on FreeBSD).
- Lightweight execution units based on greenlet.
- API that re-uses concepts from the Python standard library (for example there are Events and Queues).
- Cooperative sockets with SSL support »
- DNS queries performed through threadpool or c-ares.
- Monkey patching utility to get 3rd party modules to become cooperative
GoAccess: Pretty Web log parser that runs in the terminal
From the GoAccess website:
GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems. It provides fast and valuable
HTTP statistics for system administrators that require a visual server report on the fly.Features
- General Statistics, bandwidth, etc.
- Time taken to serve the request (useful to track pages that are slowing down your site)
- Top Visitors
- Requested files
- Requested static files, images, swf, js, etc.
- 404 or Not Found
- Hosts, Reverse DNS, IP Location
- Operating Systems
- Browsers and Spiders
- Referring Sites
- Referrers URLs
- Keyphrases
- Geo Location – Continents/Countries New
- HTTP Status Codes
- Ability to output JSON and CSV New
- Different Color Schemes
- Support for IPv6
- Unlimited log file size
- Output statistics to HTML. See report. N
Supported Formats
- Any custom log format string
- Common Log Format (CLF) Apache
- Combined Format (XLF/ELF) Apache|Nginx
- W3C format (IIS)
- Amazon CloudFront (Download Distribution).
- Apache virtual hosts
Output Formats
- Terminal (default)
- HTML
- JSON
- CSV
xip.io
This would have come in handy when I was doing a lot of testing of our mobile
apps. At work we have a “live” silo and a “dark” silo. For large deployments,
we stage the new builds in the “dark” silo and test it there. Then when it’s
time to go live with the new builds, we update the load balancers to swap the
silos and make the new code live. Pointing native apps to the dark silo
is kinda tricky since there is no /etc/hosts/
file to edit. I have been
testing by running my own DNS server on my laptop, but that is a pain. I’m
going to play with xip.io to see if it can help me with this kind of
testing.
From the website:
xip.io is a magic domain name that provides wildcard DNS
for any IP address. Say your LAN IP address is 10.0.0.1.
Using xip.io,10.0.0.1.xip.io resolves to 10.0.0.1 www.10.0.0.1.xip.io resolves to 10.0.0.1 mysite.10.0.0.1.xip.io resolves to 10.0.0.1 foo.bar.10.0.0.1.xip.io resolves to 10.0.0.1
…and so on. You can use these domains to access virtual
hosts on your development web server from devices on your
local network, like iPads, iPhones, and other computers.
No configuration required!
GitList
Another team at work has begun using GitList as their Git repository
viewer.
It looks like a nice self-hosted alternative to GitHub
From the GitList website:
GitList allows you to browse repositories using your favorite browser,
viewing files under different revisions, commit history and diffs. GitList is
free and open source software, written in PHP, on top of Silex and the Twig
template engine.
Career Advancement for Software Developers
A great blog post with advice for Software Developers:
http://peterlyons.com/leveling_up.
Scraping Data from a Website using JSoup (Lightly Seasoned with SQLite)
Here’s some Java code I wrote to scrape links from a site for insertion into a
SQLite database. I use JSoup to do the scraping. It uses JQuery selectors
to select content to scrape. I think I prefer XPath, but it’s good to
have another tool in the toolbox. I use a Google Chrome extension called
Scraper to help me form the JQuery selectors. In this example, I’m using
the SQLite FTS4 Fulltext Search extension to make the values easily
searchable in the DB.
package com.sample;
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
/**
*
* @author Oscar Grouch
*/
public class SampleScraper {
/**
* @param args the command line arguments
*/
public static void main(String[] args)
{
/* Pull the all posts page */
Document doc = null;
try
{
doc = Jsoup.connect("http://localhost/all.html").get();
}
catch (IOException ex)
{
Logger.getLogger(Scraper.class.getName()).log(Level.SEVERE, null, ex);
}
/* Get a list of all of the links on the all posts page */
Elements links = doc.select("h2 > a");
if (links != null)
{
/* Initialize DB driver. */
try
{
/* load the sqlite-JDBC driver using the current class loader */
Class.forName("org.sqlite.JDBC");
}
catch (ClassNotFoundException ex)
{
System.err.println("Class not found: org.sqlite.JDBC: " + ex.getMessage());
}
Connection connection;
try
{
/* create a database connection */
connection = DriverManager.getConnection("jdbc:sqlite:C:/Temp/sample.db");
Statement statement = connection.createStatement();
statement.setQueryTimeout(30); // set timeout to 30 sec.
statement.executeUpdate("drop table if exists content");
statement.executeUpdate("CREATE VIRTUAL TABLE content " +
"USING fts4(title, url, post)");
}
catch(SQLException e)
{
// if the error message is "out of memory",
// it probably means no database file is found
System.err.println(e.getMessage());
return;
}
/* Loop through the links */
for (Element link : links)
{
/* Print out the link text */
System.out.println("Processing " + link.text() + "...");
Document currentDoc = null;
try
{
/* Pull the current link page */
currentDoc = Jsoup.connect(link.attr("abs:href")).get();
}
catch (IOException ex)
{
Logger.getLogger(Scraper.class.getName()).log(Level.SEVERE, null, ex);
}
/* Get the post Text */
Element post = currentDoc.select("div.post").get(0);
String postText = post.text();
/* Print the post text */
System.out.println("Post Text: " + postText);
try
{
PreparedStatement preparedStatement
= connection.prepareStatement("insert into content values(?, ?, ? )");
preparedStatement.setString(1, link.text());
preparedStatement.setString(2, link.attr("abs:href"));
preparedStatement.setString(3, postText);
boolean isResultSet = preparedStatement.execute();
if (isResultSet)
{
System.out.println("A ResultSet was returned.");
}
else
{
System.out.println("No ResultSet was returned.");
}
}
catch(SQLException e)
{
// if the error message is "out of memory",
// it probably means no database file is found
System.err.println(e.getMessage());
}
}
}
}
}