I’ve set up a new repository which mirrors several scripts from my github repositories. Click the Scripts link in the main menu at the top of the page to browse these.
See the README for more details.
Tag: python
PokeyCrawl available via PyPi
Python Web Crawler
Spiders and webs
Web Spiders can be very useful to the Website Administrator, in addition to indexing your sites, you can load test the server and an intelligent web crawler can even simulate normal -> moderate -> high levels of web traffic, allowing you to benchmark your website and server performance information.
While there are potentially malicious applications for any web technology, the utility of the Web Crawler makes having one handy a good idea. Here is my implementation, using Python multiprocessing, urllib, and socket.
New mailer.py
The Python smtplib module offers a high-level interface to what is typically a very low-level process, offering handy dictionary returns and several convenience methods to process and send SMTP e-mail messages. Anyone who has worked with e-mail knows it can be a beast, with effective diagnosis of issues often involving time-consuming review of cryptic logs and a cascading list of potential “gotchas” to miss or misinterpret; this tool is part one in a suite of utilities that will help to streamline the scripted sending of messages and provide a more human-readable alerts system based on scraping e-mail headers of received messages.
New PokeyConfig Release!
In an effort to get away from custom-derived formats, I’ve retired legacy delimited files in favor of more universal JSON and YAML formats (either one!). This will require the PyYaml module to be added in most cases. Using pip :
#sudo pip install --upgrade pip #sudo pip install pyyaml
The feature list for the next major release will include some sort of dependency management.
WorldGenerator Update
The first obstacle to tackle with the PokeyGame idea was world generation – how do you generate a randomized world that can be traversed? I ultimately decided to build a procedural level generator capable of churning out levels on demand, or according to a predefined structure and saved to a template file that can be read later. This lead to the WorldGenerator class I’ll be demonstrating. This is my first iteration, I’ve decided to come up with my own method, and then investigate other methods once I am happy with mine to see how it compares.
Pathfinder explained
The Pathfinder class is used in the WorldGenerator functionality to ensure each level is passable, and I plan on integrating with the enemy AI for random level traversal and encounters.
site_backups alpha version released!
Alpha Version
The website backups utility is functioning in a limited capacity, there are few a still bugs to iron out but runs are succeeding without error and the results are as expected.
While I was intending to use the MySQLdb module to handle all database activities, I opted to use it instead only for the connectivity and diagnostic testing portion of the application. When it comes to actually dumping the data, mysqldump is a simpler (and faster in some cases) solution.
PokeyWorks Update
Check out the newly added PokeyWorks PokeyConfig description at the project page.
This class is at the core of PokeyWorks and was the initial reason I started the framework. It offers a file-based configuration superclass, enabling a familiar route to configuring an application, and the flexibility of multiple separate configurations.
See my GitHub repository, I will have a Readme available soon.
