Elasticon 2024

Tue, 28 Jan 2025 04:00:00 +0000

This is a bit of a retroactive post that’s been sitting in my drafts, but two months ago I had the pleasure of attending ElasticON 2024 in New York City thanks to our company being a partner and sponsor for the event. It was a great experience and I learned a lot about current and upcoming features within the Elastic stack. I also had the opportunity to meet some of the Elastic team as well as some of my clients.

The conference was held at the Manhattan Center, a beautiful venue in Midtown Manhattan. The trip was spread out over two days, with the first day being dedicated to keynote sessions and the second day being dedicated to workshops. GenAI was a big focus at the conference, with most of the events being centered around it in some form or another, whether it’d be RAG (Retrieval Augmented Generation) applications or getting into the nitty-gritty of the underlying technology with things like BBQ (Better Binary Quantization).

One of my favorite events was the Geospatial lightning talk where it covered building out a RAG service leveraging textual and vector data with geospatial data.

Another one of my favorites was the security presentation going over Attack Discovery, which illustrated a really cool feature of mapping out the relationship of security events to the MITRE ATT&CK framework.

The following day, we attended the Elastic office where we were able to both attend and lead workshops for GenAI certification.

Least but not least, it wouldn’t be a conference without a swag haul picture.

Overall, it was a great experience and would definitely recommend attending ElasticON if you have the opportunity.

RRTA ModeShift Reverse-Engineering

Thu, 28 Dec 2023 18:00:00 +0000

A few years ago, the RRTA (Red Rose Transit Authority) performed a big overhaul to their transit system, including the introduction of a new mobile app called RRTA Go Mobile by ModeShift. The app was designed to replace the old paper tickets and passes with a digital system that could be used to purchase passes and plan trips. As a big proponent of public transit, this was a very welcomed improvement for me. There were a few issues early on with QR codes not scanning properly and/or failing to renew, but those issues were quickly resolved. I myself have submitted a ticket or two to the developers regarding some issues I’ve encountered and received a fairly timely response with fixes following shortly after, so props to the developers for that.

RRTA also offers another app/web interface called BusFinder operating on the myStop system by Avail Tech, which allows you to track buses in real-time, a feature ModeShift sadly fails to offer. This effectively requires riders to utilize multiple apps for basic functionality and RRTA doesn’t seem to advertise the BusFinder app very well, as I’ve met many riders who were unaware of its existence.

Earlier this year, I decided to take a look at the BusFinder system and see if I could reverse-engineer it to create my own tools for tracking and analytics. I was able to get a basic prototype working, which was pretty simple considering it’s a pretty basic REST API and eventually released a Python wrapper for it.

I was curious if I could do the same with ModeShift, but I quickly discovered that it was a bit more complicated than BusFinder. The ModeShift app is a hybrid app, meaning it’s essentially a web app wrapped in a native app. This is a pretty common practice for mobile apps, as it allows developers to write a single codebase for multiple platforms, but it also makes it a bit more difficult to reverse-engineer.

I started by decoding the APK using apktool. Next I replaced the Network Security Configuration to allow user-installed certificates, which is required to intercept HTTPS traffic. The source code for the app was then modified to disable any various certificate pinning methods and to allow the use of a custom CA certificate. After that, the modified APK was recompiled with Apktool, signed with my own key via uber-apk-signer, and installed on my device.

I then used Fiddler Everywhere to act as a man-in-the-middle (MITM) proxy to intercept the HTTPS traffic from the app. There are plenty of free/open source alternatives to Fiddler, but I’ve found Fiddler to be the most user-friendly and wanted to try the Everywhere version, as I’ve been a long-time user of the original Fiddler for Windows.

I’m not particularly interested in any of the authenticated endpoints, so I’m only going to focus on the unauthenticated endpoints for now since they’re the most useful and allow stuff like trip planning. Admittedly, these endpoints do a lot of the heavy lifting as far as actual routing and trip planning goes. I was initially assuming that ModeShift was piggybacking off the same system as BusFinder (they perhaps are, to a degree), but it seems that ModeShift is doing a lot of the heavy lifting itself, rather than passing off the routing to something like Routes API from Google or something similar.

My plan is to offer a client implementation for ModeShift within the python-rrta package. My hope is to create more tools for RRTA riders to utilize and to hopefully encourage more ridership.

LCWC and Dispatch Rabbit Holes

Tue, 07 Nov 2023 19:06:00 +0000

Earlier this year, I became interested in the LCWC (Lancaster County-Wide Communications) incident page - a county-wide dispatch system that provides incident updates for all of Lancaster County, PA.

It’s very useful for residents to know what’s going on with regards to various emergency services incidents. It’s not without its flaws, however. For starters, downtime seems to be a big issue especially during peak incident periods which is less than ideal considering that’s when most people are going to be slamming the site. Additionally, there doesn’t seem to be any sort of historical data. I wanted to atttempt to resolve those issues and also add some of my own, such as real-time notification systems and integrations with various social media platforms. This is where the rabbit hole begins.

Method #1 - Web Scraping

Initially, my goal was to simply scrape the web page itself, provided it was accessible considering the aforementioned frequent downtime. The downtime issue becomes even more apparent when you start polling it regularly, even at off-peak times. There was also another major issue with scraping - the lack of a static identifier for incidents. All incident details are subject to change except for the date/time (as far as I’ve been able to decern) but considering the lack of granularity with incident timestamps, there’s nothing to prevent a conflict.

Method #2 - RSS Feed

Next I decided to look into the available RSS feed. Saviour of all our issues with no drawbacks whatsoever (foreshadowing is a literary device in which a writer gives an advance hint of what is to come later in the story). The RSS feed appears to be cached with a reasonably low TTL and doesn’t seem subject to the pitfalls of the limited Drupal execution resources for the rest of the site. Additionally, it offers a GUID for each incident, which, while purely an RSS feed relic, does help us individualize incidents. Not all that glitters is gold however. The RSS feed is missing a critical component: incident categories. Unlike the webpage, the feed provides no indication of whether an incident is tagged as Fire, Medical, or Traffic. To resolve this, we essentially need a way to extrapolate what category an incident is based off of things like incident description and the units assigned, if there any even are. This requires us to have reliable references for each of the properties what they’re correlated with, certainly doable with a fair degree of accuracy, but that requires a lot of data to use as a reference and still isn’t 100% accurate.

An example RSS feed item:

<item>
<title>MEDICAL EMERGENCY</title>
<link>http://www.lcwc911.us/lcwc/lcwc/publiccad.asp</link>
<description>WEST COCALICO TOWNSHIP; SWAMP CHURCH RD & GALEN HALL RD; PENDING; </description>
<pubDate>Mon, 06 Nov 2023 13:35:26 GMT</pubDate>
<guid isPermaLink="false">d0fc384b-9468-4907-bfc5-e444864f5597</guid>
</item>

Method #3 - ESRI ArcGIS

Around this time I realized there was more data out there, just in another format: the live incident map. It featured much more in-depth data on incidents that I’ve been able to access so far including incident numbers (which resolves the static identifier issue), priority codes, longitude/latitude, and agency. This was especially helpful because I was having to do my own geocoding based off of intersection/municipality which didn’t necessarily indicate the exact coordinates of the incident and was especially troublesome if some of that data was missing form the incident listing, effectively making that impossible. The live map is based on an ESRI ArcGIS service which luckily provides a REST API, although it’s not as straightforward as I would’ve liked when it comes to real-time information. This was my first foray into ESRI so everything was a learning experience.

Initially I thought it’d be as simple as just providing some geospatial coordinates for the county to get all the active incidents within those bounds, provided courtesy of the county GIS department (albeit in a different specification, but nice nonetheless). The first hurdle I ran into was actually requesting the incident data. Since the data is intended to be used in “layers” on the live map, each incident category has it’s own endpoint and (as far as I know) no way to just lump them into a single request. So if we want Fire and Medical incidents, we have to make 2 separate requests. The second hurdle came from dealing with the caching mechanism ESRI utilizes. This makes sense when you’re dealing with scalability and users commonly requesting the same data via the map interface, which is generally going to consist of relatively predictable geospatial coordinates. To bypass this, I initially added a bit of random “jitter” to the geospatial data - just enough to be (hopefully) unique, but not enough to impact the return dataset. This actually worked very well until I realized that the caching mechanism wasn’t just based on known parameters, but ALL parameters. So the easier/more reliable method became appending the timestamp to each request. Gotta love the pre-mature over-engineering.

So now with this ESRI method everything should be good right? Nonsense. There’s one small, but glaring issue: unit names. Both the web interface and RSS feed use fully-qualified unit names and the ESRI system uses a shorthand form for unit names. For identification sake, this isn’t a big deal but the shorthand names aren’t exactly intuitive for people, so a mapping system will likely need to be created to correlate the two such as MED865 to MEDIC 86-5. Some of them are intuitive, others are not.

Remaining Issues

Given the nature of how emergency response services work, there is some overlap with out-of-county units being dispatched to incidents if they’re close to the incident at hand. This is a problem if we want to be able to identify what agency is handling said unit and we have no reliable way of doing so without knowing the other county’s unit naming scheme and basically going through this whole process with neighbouring counties, which is a more than just a bit out of scope for this project.

I’ve basically opted to use the ESRI data as the source of truth as it’s the most comprehensive with incident details. However, it does not seem to track stuff like transfers and standby statuses, as well as out-of-county incidents.

Future Plans

Living in the city, I’m rather accustomed to the sound of sirens whirring by. It’s nice to know what might be going on around me or for family/friends that are high risk for medical emergencies. So I set out to create a proximity-based notification system for incidents in my area/around my family/friends. But first we need the data. Naturally there’s no API handy so we get to do this ye old fashioned way.

I’ve been accumulating a lot of data. All the scraping functionality has been shoved into a Python package on PyPi for anybody to use. There is a REST API for querying active and historical incidents, as well as agency information. There is also a webhook service that will fire off updates to a Cloudflare Worker which formats the data into compatible formats for a variety of platforms such as Discord, Slack, Twitter, and more. If there’s interest or boredom sets in one day, I might implement a websocket interface for real-time updates.

Privately, I’ve set up a notification service for both myself and friends/family so that if an incident occurs within a specified radius of their address, I will be notified. This has proven to be almost scarily accurate as not long ago I got a notification that there was a fire at my location yet I heard no alarms within my building. Looking outside, I saw a multiple fire response units at the building directly across the street from me. Perhaps a bit more granularity is needed.

Closing Thoughts

What started as a little weekend project turned into me deep-diving into a lot of different technologies and learning a lot of new things. I’ve learned a lot about ESRI and ArcGIS, which is a very powerful platform. I’ve also learned a lot about the county’s emergency response system and how it works. A number of helpful people who work in the field have been kind enough to answer my questions and provide insight and empathize with their frustration with the current system. Throughout this I’ve learned that there is no real sense of standardization between county-level dispatch.

I’ll probably continue to work on this project here and there and play with some data visualization like heatmaps and such. I’m also considering adding a feature to allow users to subscribe to incidents within a specified radius of an address, but that’s for another day.

The API itself is open source, so technically anybody could host an instance, but I’d recommend just using the centralized instance when possible to avoid hammering the county’s/ESRI’s servers any more than necessary.

Example REST API response:

{
	"id": "6ac7e52c-977c-47aa-8654-c0daadf4c0b5",
	"category": "Medical",
	"description": "MEDICAL EMERGENCY",
	"intersection": "S PRINCE ST / W FARNUM ST",
	"municipality": "LANCASTER CITY",
	"dispatched_at": "2023-09-12T03:55:12+00:00",
	"number": 2309013581,
	"priority": 1,
	"agency": "LANCASTER EMS (LEMSA)",
	"coordinates": {
		"latitude": 40.03403523000003,
		"longitude": -76.30727306399996
	},
	"meta": {
		"added_at": "2023-09-12T04:04:10.661851",
		"updated_at": "2023-09-12T04:04:24.051156",
		"resolved_at": null,
		"client": "ArcGISClient v0.9.2",
		"automatically_resolved": false
	},
	"units": [{
		"id": "664422e6-4593-4ba6-9bd3-2bb5626bdad5",
		"name": null,
		"short_name": "INT",
		"added_at": "2023-09-12T04:04:24.095154",
		"removed_at": null,
		"last_seen": "2023-09-12T04:04:24.095154",
		"automatically_removed": false
	}, {
		"id": "823cdbdb-c15b-41f3-9015-55e7dc5ae1d6",
		"name": null,
		"short_name": "MED",
		"added_at": "2023-09-12T04:04:24.096157",
		"removed_at": null,
		"last_seen": "2023-09-12T04:04:24.096157",
		"automatically_removed": false
	}]
}

You can find the Python parser package on PyPi as well as on GitHub.

The API itself is also open source and can be found on GitHub as well and is additionally available on DockerHub.

Google Play Developer Account Termination

Mon, 16 Sep 2019 20:10:00 +0000

Recently I had the pleasure of dealing with Google’s heavy handed approach to over-sweeping account terminations.

On September 11th, I received the following email from Google out of the blue:

This is a notification that your Google Play Publisher account has been terminated.

REASON FOR TERMINATION: Prior violations of the Developer Program Policies and Developer Distribution Agreement by this or associated accounts as outlined in previous emails sent to the registered email address(es) of the Publisher account(s).

Google Play Publisher suspensions are associated with developers, and may span multiple account registrations and related Google services.

You can visit the Developer Policy Center to better understand how we enforce Developer Program Policies. If you’ve reviewed the policy and feel this termination may have been in error, please reach out to our policy support team.

Do not attempt to register a new developer account. We will not be restoring your account at this time.

The Google Play Team

I received no warnings prior to the termination email I received nor did I have any idea as to why the account was terminated to begin with. The only email I ever received regarding this account was on August 29th, 2018 where I was notified that an app was removed from Google Play because of a Children’s Online Privacy Protection Act (COPPA) compliance requirement, which I adhered to and the app was reinstated shortly thereafter.

I submitted my appeal, explaining I had never received any warnings or occurrences prior to termination and gave some basic information regarding my packages that would be pertinent to a potential occurrence (e.g., content ratings, open-source, embedded ads).

Additionally, I decided to post over on /r/androiddev about the situation and found out I wasn’t alone and there was apparently some sort of ban wave, all of which seemed to have a similar story: no prior occurrences, no warning email, and the claim of “prior occurrences” given as the reason for termination.

The following day after submitting my appeal, I received a response:

Hi Nate,

Thanks for contacting the Google Play Team.

After further review, we’ve accepted your appeal and reinstated your account. You’ll need to sign in to your Play Console to modify and/or republish any reinstated apps to make them available on Google Play.

If the option to resubmit is not available, please try making a small change to your app’s Store Listing page. For example, you can add and remove a space at the end of your app description.

Please let me know if you have any other questions.

Regards, Samantha The Google Play Team

Several other developers I’ve talked to have received the same response with no reasons given for their termination. Any Play Store listings had to be re-submitted but that aside, re-instatetment was seamless.

I responded to the reinstatement email requesting further information on whey the account was terminated and what steps I could possibly take to avoid future potential issues.

Then on September 13th, I received a follow up email:

Hi Nate Shoffner,

On September 11, 2019, an error resulted in the unintentional ban of your developer account. We apologize for this error and assure you that it will not negatively impact your account standing. We will restore all of your app(s) to their prior state as of September 11, 2019. No action is needed on your part. We expect the restoration process to be complete by September 14, 2019.

Your restored app(s) may be reviewed by the Google Play Team, so please ensure that your app(s) are compliant with all Developer Program Policies.

If you have any questions or concerns about this process, feel free to submit a request here and we’ll do our best to respond within 1 business day.

Thank you for your understanding and for being a valued part of the Google Play ecosystem.

The Google Play Team

Finally, today on September 16th I received a response to me asking for further details as to why the account was terminated to begin withL:

Hi Nate,

Thanks again for reaching out. I believe an email was sent out to you (ticket# REDACTED) last Friday explaining the issue.

Please refer to the aforementioned email for more information and if you have any further questions please let me know.

Regards, Samantha The Google Play Team

Anybody who’s had to deal with monolithic companies like Google knows how daunting it can be when something like this happens. You feel powerless because getting in contact with an actual human seems nearly impossible. What’s worse is that, from what I’ve read, any accounts associated with a terminated account is subject to termination as well. This can cause a massive domino effect when you have several accounts working for one client/company and one of them is terminated.

Google needs to work on their transparency when it comes to their “valued” members of the Google Play ecosystem. While there is definitely a case to be made for Google needing to do maintenance for malicious apps and developers, even through automated means, there needs to be a more transparent and communicative method of dealing with them in the case of false positives. When an individual’s career and livelihood can but cut off without a warning, it’s a surefire way to get people to abandon your platform for one with better insurance.

Disable Nvidia Telemetry 1.1

Sat, 26 Aug 2017 08:13:00 +0000

Pushed out a small update to for the Nvidia Telemetry tool with a few new features:

Added: Installer and portable builds
Added: Update checking
Added: Task Scheduler options
Added: File and event logging

As per usual, you can view the source code over at GitHub.

Download

Intel Core i7-7700K Delidding

Thu, 01 Jun 2017 17:16:00 +0000

For years, dating back to the Athlon line, I had been using AMD processors for my builds and had been waiting in anticipation for Ryzen’s release, which debuted in February of this year. I had been using an overclocked FX-8350 for a few years, which meant there was no upgrade path in sight that didn’t require a full architecture and socket change, whether that be AMD or Intel.

In the end, I eventually decided to go the Intel route with the i7-7700K, valuing the additional gaming performance over Ryzen’s multi-threaded capabilities - ones I wouldn’t realistically take advantage of on a day to day basis. Simultaneously, I also decided to order a delidding kit and the necessary supplies to delid the i7-7700K.

What is Delidding?

Dating back to around Haswell, Intel’s processors have been notorious for reaching extremely high temperatures, especially while overclocking. The leading issue for this seems to be due to the TIM (thermal interface material) they’re using on the die. “Delidding” is the process of removing the Integrated Heat Spreader (IHS) from a CPU with the intention of applying aftermarket TIM to the CPU. It’s unlikely that they’re merely skimping out and using ill-suited TIM, but rather focusing on longevity at stock/moderate voltages. With that in mind, it’s possible that aftermarket TIM solutions may require additional delidding at some point to re-apply fresh TIM, but that’s purely my speculation.

This isn’t going to act as in in-depth overclocking/delidding guide, there are more than enough of those already throughout the internet.

Build Specs

CPU: Intel Core i7-7700K
Cooler: Corsair H115i Liquid Cooler
Mobo: ASUS ROG MAXIMUS IX HERO
RAM: Corsair Vengeance 32GB RGB LED (4x8GB) DDR4 3000 MHZ
PSU: Corsair RM750

Benchmarks and Stress Tests:

x264 (16 threads)
Prime 95 27.9
ROG RealBench
AIDA64
Cinebench R15
Vegas PRO 14 (~5m Sony AVC 1080p60 render)

Prior to delidding, I was able to overlock to 5.0GHz @1.36v using an AVX offset of -2 and load-line calibration (LLC) set to 3. Even with the offset, AVX workloads resulted in the package hitting 100C temps and instantly shutting down. 4.9GHz @1.35v resulted in much more stability, but not by much in regards to thermals, which were still exceeding 94C and over in the long term.

Delidding Materials:

Intel Core i7-7700K (or whatever LGA 1155/1150/1151 CPU you have)
Rockit 88
Re-Lid Kit - 1150 / 1151
Thermal Grizzly Conductonaut (Coollaboratory Liquid Ultra also works)
Loctite Super Glue - Gel Control
91% Isopropyl alcohol
Coffee Filters
Cotton Swabs
Scotch Tape

Alternatively, you could you use a silicon-based gasket maker instead of the super glue, either one is easy enough to de-lid and does the job. Just note that the gasket maker will take much longer to cure, hence why I chose the super glue.

Using the Rockit88, I popped the IHS off the wafer and removed the old TIM and sealant. Just to be safe, I masked the CPU with tape prior to applying the new TIM, as the liquid metal is extremely runny, almost like liquid mercury.

Using the relid kit, I aligned the IHS back above the CPU and applied a drop of superglue on the bottom corners of the IHS and pressed it down. After allowing it to cure for 2 hours, I popped the CPU out of the kit, threw it in the socket and started it up. Almost immediately I noticed a huge difference in thermals. Under both AVX and non-AVX loads, I was barely hitting 72C which is simply mind blowing. With that headroom, I was able to bump up the voltage back to 1.36V and hit that sweet 5.0GHz

As of posting this, everything has been running stable for a few weeks now and thermals haven’t changed nor have there been any signs of instabilities.

Conclusion and Disclaimer

Overall, the delidding process is pretty simple and safe if done properly. If you’re on the fence about delidding and plan on overclocking or have started but are exceeding safe thermals, I’d definitely recommend it.

Just be aware of the risk you’re tasking when delidding your CPU is it will instantly void your warranty. Generally speaking, I’d say it’s a fairly easy process if you’re patient and do it properly.

Disable Nvidia Telemetry

Fri, 12 May 2017 20:13:00 +0000

Back in early November, Nvidia started bundling telemetry services with their drivers. Gamers Nexus did an analysis where they inspected data transactions via Wireshark. Their focus seemed to be moreso on dispelling the rumors of personally identifiable information being sent unencrypted, as they didn’t decrypt any of the encrypted traffic. Their results yielded from an hour of testing that nothing was being sent, the services were seemingly dormant.

For me, however, it’s a moot point. Nvidia should’ve been upfront about the telemetry with their users and I’m not really a fan of unnecessary services running on my computer, so naturally I disabled the services from their inception. I soon found out that they were being reactivated upon updates, which resulted in me throwing together a small utility to handle the disabling work for me.

I’ve shared it with a few people and figured I might as well throw it online.

Note: I use a small subset of GeForce Experience’s features, including ShadowPlay and have experienced no problems. I can’t speak for every use case and how disabling these services might be impactful in the future, so use it at your own risk.

You can download the utility here as well as find the source code on GitHub.

Seachem Doser 1.2

Mon, 07 Nov 2016 21:03:00 +0000

Seachem Doser 1.2 is now available on the Play Store.

Major changes include metric support, new products (see below), and an interface redesign with improved material design.

Previously, I was having to resort to reverse-engineering Seachem’s Flash-based dosage calculators in order to obtain dosage calculations. Thankfully, they finally added web-based dose calculators, albeit with some errors/miscalculations on their end at the time of writing this.

Additionally, support for Honeycomb and below has been dropped due to insignificant user distribution and the effort required to provide backwards compatibility. Sorry for the few of you that this may affect.

Full changelog:

Added: Discontinued products toggle
Added: Dosage copying
Added: Seachem warnings
Added: Seachem-recommended parameter values
Added: Pinnable products
Added: Light/Dark theme selection
Added: Flourite Sand
Added: Flourish Trace
Added: Reef Fusion 1 & 2
Added: Small (3.5 kg) gravel/sand bag dosage
Added: Metric/imperial settings
Added: Default product settings
Fixed Reef Advantage Calcium dosage
Fixed: Gray Coast calculation
Fixed: Meridian name typo
Fixed: Full-screen keyboard on landscape orientation
Changed: All gravel values now round up to next bag(s)
Changed: UI redesign (w/ improved material design)

Tablet:

Phone:

Moved to GitHub Pages & Site Redesign

Fri, 11 Mar 2016 19:41:00 +0000

I’ve just finished moving my website over from WordPress to GitHub Pages. It’s something I’ve been meaning to do for a while now. Despite using WordPress for years, I’ve found it to be rather excessive for my personal needs as far as resource usage and maintenece goes.

Although I never personally had any security issues with WordPress, it’s definitely a high-profile target and switching to static content definitely helps mitigate some attack vectors. Not to mention the inherent load time decrease because of said static content.

Having already used Jekyll/GitHub Pages many times before, the move wasn’t terribly painful. There were a few ‘gotchas’ here and there, but with a little hacking, things were ironed out. I’ll probably post some of those solutions in later posts.

I took advantage of the migration to redesign the site and perform some general maintenece on posts and the like.

Additionally, all comments have been moved to Disqus.

There may still be a few quirks here and there that I haven’t noticed yet, but overall things are working smoothly.

Seachem Doser 1.1

Wed, 23 Dec 2015 21:35:00 +0000

So I’m about 8 months late to posting this. Granted, I’m quite behind on posting a lot of developments (sleep deprivation puns). Anyway, Seachem Doser is an Android app that I developed a while ago that serves as a “dosage/quantity calculator” for various products Seachem offers.

Seachem, for those who don’t know/have lives, is a company that offers various products geared towards maintaining freshwater and marine aquariums. Personally, I use a lot of their products for my high-tech planted aquariums.

They offer calculators on their site that allow you to calculate how much you should dose specific fertilizers, buffers, conditioners, the amount of gravel needed per a specific volume, etc. Sadly, these calculators are all Flash-based. So I decided to make one that’s a bit more flexible. I should note that all of the values for these calculations were pulled from their official calculators, after a bit of reverse-engineering. At first I just made a Windows-based calculator/library that you can find on GitHub. Eventually the issue of convenience took hold and I decided to make it mobile.

As of today, I just updated it to work on tablets and polished things up a bit since I had some free time.

As with the Windows version, the Android version is also open source on GitHub.

If you happen to be somebody who will find it useful, feel free to download it through the Play Store.

Tablet

Phone:

Dropbox Does Not Access Files Throughout Your PC

Tue, 03 Mar 2015 17:05:00 +0000

Recently, there has been an article circulating around with a pretty damning accusation. To summarize the post, they claim Dropbox is transferring data on the entire computer, not just folders that are designated to be synced.

This is clearly a bold accusation, with seemingly little evidence to back it. This speculation is based entirely on filesystem events and unverified network traffic..yep.

At the time of writing this, the author has not disclosed what “unnamed DLP agent” software he is using nor has he provided any real evidence to support his claims.

Let’s explain the filesystem activity:

Just to run a simple test, I’ll be using Process Monitor and setting basic filters to check Explorer and Dropbox filesystem events.

After creating a new file outside of my designated Dropbox directory, you can see it fires filesystem events to query the file. In particular, QueryDirectory operations which involve checking the file path to see if it belongs to a synced directory.

I’d expect another culprit to the the shell integration. Personally, I don’t use the sync indicators myself, as I have other programs that use shell integration and it can cause conflicts. However, I’d imagine that the shell integration in Explorer requires the files/directories to be queried as well to check for sync status.

As for the network activity…the author has not even bothered to check the length of the data and compared it to the suspected files nor has he compared the timestamps between the two. Keep in mind that network activity from a program such as Dropbox is perfectly normal, how else would it be able to sync things in realtime?

So no, based off the limited (see: hardly any) “evidence” provided by the original author, it does not appear that Dropbox is transferring data outside of the designated dirctories.

"Pretend You're Xyzzy" Server Setup and Build Instructions

Sun, 15 Feb 2015 18:12:00 +0000

“Pretend You’re Xyzzy” is a Cards Against Humanity clone that allows you to play online with others. It consists of a Tomcat/JSP server and web page clients.

At the time of writing this, the main server located at pretendyoure.xyz appears to be down, so I decided to set up my own server for a group of friends and myself to play. Initially, there didn’t seem to be much info on setting up the server. The GitHub had a very brief explanation:

This project has only been tested in Tomcat 7 and is known to not work in Tomcat 6 without some finagling. Currently, the only automated way to build is is using the Eclipse project.

While the setup isn’t terribly complicated or anything, I figured I’d post some instructions on how to set things up since a lot of people seem to be having issues doing so and there don’t seem to be any concise instructions on how to do so.

These instructions are for a Debian-based installation, but are easily applicable to other distros/platforms as well.

First you’ll need Apache Tomcat 7 and you’ll want to install the admin panel as well:

sudo apt-get install tomcat7 tomcat7-admin

Next, stop the Tomcat server for the time being so we can modify some permissions:

service tomcat7 stop

Next, we’ll edit the users config so we can administrate the server:

sudo nano /etc/tomcat7/tomcat-users.xml

Add the following line in between the and tags, replacing ‘admin’ and ‘password’ with your own secure credentials:

<user username="admin" password="password" roles="manager-gui,admin-gui"/>

Start the service back up:

service tomcat7 start

Now we want to install PostgreSQL for our database:

sudo apt-get install postgresql

Change over to the postgres user (already set up via the installation process):

sudo -i -u postgres

Then, run the following, replacing ‘password’ with a secure password of your choice:

createdb cahdb
psql template1
CREATE USER cah WITH PASSWORD 'password';
GRANT ALL PRIVILEGES ON DATABASE "cahdb" to cah;
\q
exit

Finally, we need to download the cards database and import it:

wget https://raw.githubusercontent.com/ajanata/PretendYoureXyzzy/master/cah_cards.sql
psql -h localhost -d cahdb -U cah -f cah_cards.sql

Now navigate to http://your_host:8080/manager/html and login.

From here, you can upload your WAR file and deploy it.

If your server does not seem to be loading/seems to be stalling, a possible reason could be Tomcat using a blocking entropy source for session IDs. To implement a non-blocking (and slightly less secure) entropy source, open/create a setenv.sh script in /usr/share/tomcat7/bin and add the following line:

JAVA_OPTS=" $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom "

Save and restart Tomcat.

service tomcat7 restart
Finally, you can access the app via http://your_host:8080/cah.

You can download a pre-built/zipped version of cah.war here. The PostgreSQL parameters are as follows:

Username: cah
Password: password
Database: cahdb

NOTE: Since this version uses insecure/public credentials, make sure to take necessary precautions to secure your server.

If you’d prefer to build your own version, follow the instructions below.

Building the .war file is fairly straightforward. Just download the source from Github via Git or just use the zip functionality.

You will need the JRE v7u75 installed.

Open Eclipse (Java EE) and open the project.

Install Luna if you don’t already have it buy going to Help->Install New Software. Choose “Luna”. Select “Web, Xml, Java EE and OSGi Enterprise Development”, install it and restart.

Set Tomcat 7 as your runtime environment server.

Make sure your build path uses the correct runtime environment as well.

Database credentials can be changed in the hibernate.cfg.xml file.

When all is well, you can go to File->Export->Web->WAR File.

You can upload and deploy that file through the Tomcat admin panel and you’re good to go!

Programming - Paradigms, Satire, and Precedents

Sat, 14 Dec 2013 12:28:00 +0000

I’m not normally one to get absorbed into drama or gossip on the internet, but this evening I found myself a bit agitated, but not for the reason you might initially assume.

Introducing C+=

C+= (pronounced either C-plus-Equality, or See Equality) is a feminist programming language, created to smash the toxic Patriarchy that is inherent in and that permeates all current computer programming languages. Inspired by the <a href="http://www.hastac.org/blogs/ari-schlesinger/2013/11/26/feminism-and-programming-languages" target="_blank">ground-breaking feminist research of Arielle Schlesinger</a>.

C+= was created by the Feminist Software Foundation, whose language philosophy entails the following:

The language is to be strictly interpreted using feminist theory. Under no circumstances should the language be compiled, as compilation and the use of a compiler imposes an oppressive and toxic relationship between the high-level descriptive language and the low-level machine code that does all the labo(u)r. Instead, C+= is interpreted, which fosters communication, itself a strong female trait.

No constants or persistence. Rigidity is masculine; the feminine is fluid. I.e., fluid mechanics is hard for men 'because it deals with "feminine" fluids in contrast to "masculine" rigid mechanics'.

No state. The State is The Man. 'Nuff said. Hence, the language should be purely functional.

Women are better than men with natural language. Hence, the language should be English-based like HyperCard/LiveCode.

No class hierarchy or other stigmata of OOP (objectification-oriented programming). In fact, as an intersectional acknowledgement of Class Struggle our language will have no classes at all.

On the off chance that objects do mysteriously manifest, there should be no object inheritance, as inheritance is a tool of the Patriarchy. Instead, there will be object reparations.

Societal influences have made men often focus on the exterior appearances of women. This poisons our society and renders relationships to be shallow, chauvinistic, and debases our standards of beauty. To combat that, C+= is to tackle only audio and text I/O, and never graphics.

Unicode is the preferred character encoding due to its enabling the diverse aesthetic experiences and functionality that is beyond ASCII. UTF-8 is the encoding of choice for C+=.

Women are more social than men. Hence, social coding should be the only option. The code only runs if it is in a public repo.

Instead of "running" a program, which implies thin privilege and pressure to "work out", programs are "given birth". After birth, a program rolls for a 40% chance of executing literally as the code is written, 40% of being "psychoanalytically incompatible", and 40% of executing by a metaphorical epistemology the order of the functions found in main().

Programs are never to be ["forked"](https://en.wikipedia.org/wiki/Fork_(system_call)), as the word has clear misogynistic tendencies and is deeply problematic. Instead, programmers may never demand "forking", but ask for the program to voluntarily give permission. "Forking" will henceforth be called "consenting", and it is entirely up to the program to decide if the consent stands valid, regardless of the progress of the system clock.

Forced program termination is not allowed unless the program consents to it. The process is part of the choice of the program, not the programmer.

Licensing: the Feminist Software Foundation License.

Here is some example FizzBuzz code written in C+=:

#consider <FEMINIST_RAGE.Xir>;

// TODO replace main() as Progrym entry point; "structured
// programming" is classist oppression
// TODO2: main() is now womain()

// NB one does not argue with C+= Progryms; one makes requests, which
// the Progrym is free to consider or ignore as she pleases
xe womain (xe RequestCount, strong *RequestList[]) {
// NB typically patriarchal fizzbuzz enshrines socially
// constructed limits as immutable fact; here we expose this
// subtle mental tyranny for what it truly is

xe ArbitraryBeginning accepts(present(-50));
xe ArbitraryEnd accepts(present(50));

// naturally, everything revolves around this
xe ThePlaceBetween accepts(present(0));

among(ThePlaceBetween accepts(ArbitraryBeginning),
      ThePlaceBetween honors(ArbitraryEnd),
      ThePlaceBetween improvesBy(present(1))) {
    check(ThePlaceBetween envelops(present(3))) {
        yell(present("Fizz"));
    }
    recheck(ThePlaceBetween envelops(present(5))) {
        yell(present("Buzz"));
    }
    unpack {
        // strength &amp; independence!
        yell(present(ThePlaceBetween));
    };

    yell(present("\n"));
};
present(Satisfaction); };

By now, I hope you would realize that this is in fact satirical. Personally, I found the whole thing quite humorous and well-constructed in regards to the amount of effort and detail put into it, code examples and all. Now I know what you’re thinking. “Gee, how will satire mocking feminism be perceived on the internet?”. This is where things get interesting…

The project was initially hosted on Github, that is until Github decided to pull it due to harassment complaints. Who was being harassed? Nobody. At least not by the project owners. It seems that there were various people, namely feminists, who were being impersonated and had commits to the project being sent under their fraudulent identities. Maybe I’m missing something, but it seems just a tad bit excessive to close the entire repository for an incident caused by a few select individuals who didn’t even own the repository. The project eventually moved to BitBucket and the complaints soon followed, just as baseless and shallow as before. There were even individuals trying to get the domain dropped by CloudFlare. It became quite apparent that these people were acting on emotion and not any sort of actual logic. Naturally, as with any ethical “debate”, there is a astounding amount of hypocrisy and generalizations that just point out the inconsistencies (or possible similarities) between the two parties.

Some people lack the ability to separate a difference of opinion and actual harassment. Pointing fingers at the wrong people (the repository owner) just makes it worse. Just because you don’t agree with the project ethically is no excuse for blatant censorship, it doesn’t matter how to try to spin it. If you do succeed with censoring the other party, you should be aware of the Streisand effect. This becomes even more crucial when you consider that the project was spawned out of 4chan.

Overall, I am quite disappointed in Github for the way that they handled (or mishandled rather) this situation. In an attempt to save face, they jumped the gun and quite possibly set a precedent for similar situations in the future. A repo owner should not have to do background checks on each individuals wanting to commit. The entire situation is a result of hypersensitivity and some people’s inability to take into consideration that their outlash could have real world effects. An incident involving gendered pronouns a few weeks back was depressingly similar.

It’s quite obvious that this project is misogynistic, satirical or not, but that’s not the problem I have with it. My issue with this whole situation is the blatant censorship and corrosive mindset of “I don’t like this, take it down” that constantly plagues the internet. People need to learn to value free speech over their own ideals, otherwise we’re looking at a very bleak future. Attempting to censor those with different opinions will only promote chilling effects, encourage polarization of those involved, and further degrade any sense of intelligent discussion there may be.

As of writing this, BitBucket has not removed the repository. Jesse Yowell from Atlassian has stated:

We are aware of this and it is currently being reviewed by our legal team.

Hopefully they will properly think this through, not just for public relations sake, but for the long-term effects it could pose. It’d be a shame to see them make the same mistake as Github.

C+= Sources:

Homepage

BitBucket Repo

Update 12/15/2013

Erik van Zijst of Atlassian has stated that they will not be removing the repo:

We have no intentions to censor at this point.

Update 12/19/2013

Contradicting the previous statement, the repo has been removed (along with forks, both public and private). Scott Farquhar has commented stating:

After further consideration, we have decided to remove this repository. While our End User Agreement explicitly prohibits the posting of content that is "racially or ethnically offensive," we believe it is consistent with the spirit of our agreement to also prohibit content that is offensive toward a specific gender. We will update our End User Agreement to make this prohibition more explicit.

Great job, updating your end user agreement for the sake of saving face.

Programming as an Essential Skill Set

Tue, 10 Dec 2013 13:23:00 +0000

Lately, the internet has been abuzz with the recent “Hour of Code” campaign. For those unaware, this is a campaign that aims to introduce people (largely children), to the world of programming in order to “jump-start their futures”. The campaign features well-renowned programmers such as Ashton Kutcher, Shakira, and Barack Obama. Now I’m not attempting make light of Obama’s apparent bubble sort knowledge, but it just feels far too superficial. Additionally, I’m not necessarily saying that these individuals don’t know or can’t learn how to program, but all of these campaigns over years have been the same. You grab a few celebrities who quite obviously aren’t interested in this sort of thing and try to sell it to the audience. It’s quite painful for me to watch.

A while back, Jeff Atwood wrote a nice articleon a subject very similar regarding CodeAcademy’s New Year’s resolution “Code Year”. That article is directly applicable to this situation as well for the most part.

Initially as a programmer, it’s easy to assume everybody should learn to code. After all, it’d make the world a better place right? Surely programming must be an essential skill set at this point that everybody should have?

No. It’s a valuable skill set, but not a truly essential one.

The real value of programming

Programming is many things, but at the end of the day it boils down to solving problems. It’s not about memorizing syntax and bashing out code, there’s a thought process behind it. Truly learning to program will give you insight on a whole new way to analyze and solve problems. In the process of learning to code, you will in turn gain a better understanding and respect for the various levels of technology upon which you are working with. It could help you more easily debug a problem you are having with say your computer or smart phone. Relatively mundane tasks can be simplified in day to day situations with just a little bit of programming knowledge.

While I can understand the thought process behind teaching everybody how to program, I am not convinced that it is necessary. I think schools shouldn’t necessarily be focusing on teaching children to program, but rather teach them some general computer/internet concepts (typing classes do not count). That would be far more valuable to the average person compared to teaching them what a while loop is.

Growing up, I didn’t have any school courses that pertained to programming (or computers at all) although I was very interested in the subject. Had they been offered, I would’ve immediately signed up.

If you have even the slightest interest in learning to code, do it. You will need to have patience and be persistent about it. It’s not something that happens overnight. If you really feel as though it’s not for you, that very well may be the case. However, if it is something that are genuinely interested in, here are a few tips:

Be patient. Trial and error can be the best instructor at times
Try to work on something you are truly passionate about
Don’t obsess over the language, framework, or platform
Google is your friend, but don’t become too dependent

Programming is not for everybody. Don’t force people, especially children, to learn to code. If you don’t have a passion to code, it will be a long and strenuous process. By forcing them to learn it, it could very well turn them off from it entirely. Let them be exposed to it on their own volition by providing the necessary tools and guidance should they be interested.

Spadille 1.6 - Better Late Than Never?

Wed, 30 Oct 2013 14:31:00 +0000

Well, over a year and a half later, I suppose I should release Spadille 1.6 eh?

Spadille dropped off the grid a bit after being put on the back burner a while ago. I stopped working on it during the middle of refactoring…which is a bad idea. It’s been ready for release for a long time now, but I just never released it. Why do today what you can put off until tomorrow? This version does not support OpenSpades yet, but I will release a quick patch shortly for it.

You can download Spadille 1.6 here.

Changelog:

Countless minor UI changes
Fixed a bunch of minor bugs
Improved memory usage
Improved threading
Rewrote screenshot hooking, caching and manager
Added dynamically populated server filters
Added share link
Added autocomplete to server browser
Added mumble link
Removed deprecated dependencies
Fixed support for Build and Shoot
Abstracted Spadille client and SpadilleDotNet library
Added “Last Played” column
Added intermediate dialog when joining server

I’ll likely be releasing my library that Spadille is based on (along with a few other projects), SpadilleDotNet. It’s a C#/.NET (2.0 compliant) library that provides a lot of functionality for working with Ace of Spades Classic (Voxlap/OpenSpades), Build and Shoot, Ace of Spades 1.0, and more.

Additionally, I have a few more tools that I’ll be releasing here soon as well, as I start to dig through old projects that I’ve not released, so keep an eye out.

C# - Global Hotkeys via GetAsyncKeyState

Sun, 13 Oct 2013 20:44:00 +0000

Here’s a quick little class for implementing global hotkeys in C#/.NET 2.0+.

As a bit of a preface, you could easily use RegisterHotKey and UnregisterHotKey respectively to accomplish something like this, but this has a few caveats:

You can’t register a key that has already been previously registered.
Some keys are reserved and cannot be registered, such as F12.
Although hacks exist, it’s not very ideal to easily implement with console applications due to the lack of a proper window handle.

Alternatively, we can use another WinAPI function: GetAsyncKeyState. It requires a bit more manual work, but it’s simple in the end.

Note: Without getting into semantics about what constitutes a proper “hook”, I’m just going to refer to the process of monitoring/polling key events via GetAsyncKeyState() as “hooking”.

To keep things simple, the actually hooking process will use the Keys Enumeration and will marshall their underlying integral type during the P/Invoke. When a key is pressed, we will trigger the KeyPressed event.

Modifier Keys

We don’t want to limit the hooking to just basic keys, instead will allow optional modifier keys using the ModifierKeys enum.

[Flags]
public enum ModifierKeys : uint
{
    None = 0,
    Alt = 1,
    Control = 2,
    Shift = 4,
}

Hooking & Unhooking

To hook a key, call the Hook() method, supplying the Keys value as well as any optional modifier keys. Additionally, you can provide a delegate to use for a callback for when the key is pressed.

To unhook a key, simple call the Unhook() method with the appropriate parameters and it will no longer be polled.

[Flags]
var keyboard = new KeyboardHook();
keyboard.Hook(Keys.PrintScreen);
 
// do something
 
keyboard.Unhook(Keys.PrintScreen);

Internally, the hooked keys will be stored as KeyHook objects, which provide Keys and Modifiers properties.

private class KeyHook
{
    public KeyHook(Keys key, ModifierKeys modifiers, KeyHookDelegate pressed = null)
    {
        Key = key;
        Modifiers = modifiers;
        Pressed = pressed;
    }
 
    public Keys Key { get; private set; }
    public ModifierKeys Modifiers { get; private set; }
    public KeyHookDelegate Pressed { get; private set; }
}

Prioritization

For simple hotkeys, there won’t be any collisions. However, when you start mixing and matching modifier keys, things can get a little messy. To alleviate this issue, the hooked keys are sorted internally using a custom IComparer<T>:

private class HookComparer : IComparer<KeyHook>
{
    private static int GetModifierCount(ModifierKeys modifiers)
    {
        var iValue = (int) modifiers;
        var bitCount = 0;
 
        while (iValue != 0)
        {
            iValue = iValue & (iValue - 1);
            bitCount++;
        }
 
        return bitCount;
    }
 
    #region Implementation of IComparer<KeyHook>
 
    public int Compare(KeyHook x, KeyHook y)
    {
        var keyCompare = x.Key.CompareTo(y.Key);
        if (keyCompare != 0)
            return keyCompare;
 
        var modifierCount = GetModifierCount(y.Modifiers).CompareTo(GetModifierCount(x.Modifiers));
        if (modifierCount != 0)
            return modifierCount;
 
        return ((int) x.Modifiers).CompareTo((int) y.Modifiers);
    }
 
    #endregion
}

Basically, it just compares hooked keys based on the following:

Keys enumeration
Number of ModiferKeys set
Underlying ModifierKeys integral type summation

To sort the list, we use a simple lambda expression with our comparer:

_keys.Sort((k1, k2) => new HookComparer().Compare(k1, k2));

Polling

The underlying polling is based on a SystemTimer.Timer. According to official Microsoft sources, this has a resolution of 15.6ms:

The default timer resolution on Windows 7 is 15.6 milliseconds (ms). Some applications reduce this to 1 ms, which reduces the battery run time on mobile systems by as much as 25 percent.

I’m not about to perform a case steady on how fast a human can realistically type versus the timer interval, configure the interval as necessary. The polling itself can be enabled/disabled via the Enabled property. Additionally, polling is suppressed when hooking/unhooking keys. During each tick, the key states are polled via PollKeyStates():

private void PollKeyStates()
{
    var altPressed = Convert.ToBoolean(GetAsyncKeyState(Keys.Menu));
    var controlPressed = Convert.ToBoolean(GetAsyncKeyState(Keys.ControlKey));
    var shiftPressed = Convert.ToBoolean(GetAsyncKeyState(Keys.ShiftKey));
 
    var pressedKeys = new List<Keys>();
 
    foreach (var key in _keys)
    {
        if ((GetAsyncKeyState(key.Key) == -32767))
            pressedKeys.Add(key.Key);
    }
 
    foreach (var key in _keys)
    {
        if (!pressedKeys.Contains(key.Key))
            continue;
 
        if ((key.Modifiers & ModifierKeys.None) == ModifierKeys.None)
        {
            if ((key.Modifiers & ModifierKeys.Alt) == ModifierKeys.Alt && !altPressed)
                continue;
            if ((key.Modifiers & ModifierKeys.Control) == ModifierKeys.Control && !controlPressed)
                continue;
            if ((key.Modifiers & ModifierKeys.Shift) == ModifierKeys.Shift && !shiftPressed)
                continue;
        }
 
        pressedKeys.Remove(key.Key);
 
        if (KeyPressed != null)
            KeyPressed(this, new KeyPressedEventArgs(key.Modifiers, key.Key));
 
        if (key.Pressed != null)
            key.Pressed(this, EventArgs.Empty);
    }
}

When using GetAsyncKeyState() you need to pay attention to the most and least significant bits:

If the function succeeds, the return value specifies whether the key was pressed since the last call to GetAsyncKeyState, and whether the key is currently up or down. If the most significant bit is set, the key is down, and if the least significant bit is set, the key was pressed after the previous call to GetAsyncKeyState. However, you should not rely on this last behavior.

Normally when polling like these, we would end up triggering multiple KeyPressed events within a few milliseconds apart from each other. We can use the return value to get around this.

A temporary List is created which contains all hooked keys which are currently pressed. This way we don't need to call GetAsyncKeyState() while iterating over the hooked keys and we can prevent hook collisions. This is where the key sorting from earlier comes into play. We don't want to prematurely trigger a KeyPressed event for a different hooked key than expected.

This isn’t necessarily a perfect solution but it works and is simple and flexible.

Finally, here’s the complete class:

/*
* KeyboardHook.cs by Nate Shoffner
* http://nateshoffner.com
*/
 
#region
 
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Timers;
using System.Windows.Forms;
using Timer = System.Timers.Timer;
 
#endregion
 
namespace GlobalHotkeys
{
    [Flags]
    public enum ModifierKeys : uint
    {
        None = 0,
        Alt = 1,
        Control = 2,
        Shift = 4,
    }
 
    public class KeyPressedEventArgs : EventArgs
    {
        internal KeyPressedEventArgs(ModifierKeys modifiers, Keys key)
        {
            Modifiers = modifiers;
            Key = key;
        }
 
        public Keys Key { get; private set; }
        public ModifierKeys Modifiers { get; private set; }
    }
 
    public class KeyboardHook : IDisposable
    {
        #region P/Invokes
 
        [DllImport("user32.dll")]
        private static extern short GetAsyncKeyState(Keys vKey);
 
        #endregion
 
        #region Delegates
 
        public delegate void KeyHookDelegate(object sender, EventArgs e);
 
        #endregion
 
        private readonly List<KeyHook> _keys = new List<KeyHook>();
        private readonly Timer _timer;
 
        public KeyboardHook()
        {
            _timer = new Timer {Interval = 75};
            _timer.Elapsed += _timer_Elapsed;
        }
 
        public bool Enabled
        {
            get { return _timer.Enabled; }
 
            set
            {
                if (value)
                    _timer.Start();
                else
                    _timer.Stop();
            }
        }
 
        public event EventHandler<KeyPressedEventArgs> KeyPressed;
 
        public bool Hook(Keys key, ModifierKeys modifiers = ModifierKeys.None, KeyHookDelegate pressed = null)
        {
            if (_timer.Enabled)
                _timer.Stop();
 
            var exists = _keys.Find(x => x.Key == key && x.Modifiers == modifiers) != null;
 
            if (!exists)
            {
                _keys.Add(new KeyHook(key, modifiers, pressed));
                _keys.Sort((k1, k2) => new HookComparer().Compare(k1, k2));
            }
 
            if (_keys.Count > 0)
                _timer.Start();
 
            return !exists;
        }
 
        public bool Unhook(Keys key, ModifierKeys modifiers = ModifierKeys.None)
        {
            if (_timer.Enabled)
                _timer.Stop();
 
            var i = _keys.FindIndex(x => x.Key == key && x.Modifiers == modifiers);
 
            if (i >= 0)
            {
                _keys.RemoveAt(i);
                return true;
            }
 
            if (_keys.Count > 0)
                _timer.Start();
 
            return false;
        }
 
        private void _timer_Elapsed(object sender, ElapsedEventArgs e)
        {
            PollKeyStates();
        }
 
        private void PollKeyStates()
        {
            var altPressed = Convert.ToBoolean(GetAsyncKeyState(Keys.Menu));
            var controlPressed = Convert.ToBoolean(GetAsyncKeyState(Keys.ControlKey));
            var shiftPressed = Convert.ToBoolean(GetAsyncKeyState(Keys.ShiftKey));
 
            var pressedKeys = new List<Keys>();
 
            foreach (var key in _keys)
            {
                if ((GetAsyncKeyState(key.Key) == -32767))
                    pressedKeys.Add(key.Key);
            }
 
            foreach (var key in _keys)
            {
                if (!pressedKeys.Contains(key.Key))
                    continue;
 
                if ((key.Modifiers & ModifierKeys.None) == ModifierKeys.None)
                {
                    if ((key.Modifiers & ModifierKeys.Alt) == ModifierKeys.Alt && !altPressed)
                        continue;
                    if ((key.Modifiers & ModifierKeys.Control) == ModifierKeys.Control && !controlPressed)
                        continue;
                    if ((key.Modifiers & ModifierKeys.Shift) == ModifierKeys.Shift && !shiftPressed)
                        continue;
                }
 
                pressedKeys.Remove(key.Key);
 
                if (KeyPressed != null)
                    KeyPressed(this, new KeyPressedEventArgs(key.Modifiers, key.Key));
            }
        }
 
        #region Nested type: HookComparer
 
        private class HookComparer : IComparer<KeyHook>
        {
            private static int GetModifierCount(ModifierKeys modifiers)
            {
                var iValue = (int) modifiers;
                var bitCount = 0;
 
                while (iValue != 0)
                {
                    iValue = iValue & (iValue - 1);
                    bitCount++;
                }
 
                return bitCount;
            }
 
            #region Implementation of IComparer<KeyHook>
 
            public int Compare(KeyHook x, KeyHook y)
            {
                var keyCompare = x.Key.CompareTo(y.Key);
                if (keyCompare != 0)
                    return keyCompare;
 
                var modifierCount = GetModifierCount(y.Modifiers).CompareTo(GetModifierCount(x.Modifiers));
                if (modifierCount != 0)
                    return modifierCount;
 
                return ((int) x.Modifiers).CompareTo((int) y.Modifiers);
            }
 
            #endregion
        }
 
        #endregion
 
        #region Implementation of IDisposable
 
        public void Dispose()
        {
            _timer.Stop();
            _timer.Dispose();
        }
 
        #endregion
 
        #region Nested type: KeyHook
 
        private class KeyHook
        {
            public KeyHook(Keys key, ModifierKeys modifiers)
            {
                Key = key;
                Modifiers = modifiers;
            }
 
            public Keys Key { get; private set; }
            public ModifierKeys Modifiers { get; private set; }
        }
 
        #endregion
    }
}

Tabster 1.6 Released

Thu, 29 Aug 2013 11:02:00 +0000

Random update time! I have a bad habit of not updating my applications for long periods of time…

A few weeks ago, I received a pingback from AddictiveTips where they wrote an articleon Tabster. I’m still not sure why I just now received the PingBack when the article was published on March 29th of this year. This isn’t the first time Tabster has been reviewed by any means (and has also been used in schools), but it brought the neglected project to my attention.

Well, much re-factoring later and Tabster 1.6 is ready.

Change Log:

Added “My Downloads” section
Added auto-scrolling
Added single-instance functionality
Added persistent window size/state
Added preview pane to library and search
Updated file formats
Caching is now forced
Removed auto-detect
Removed web browser
Fixed and improved searching
Added multi-downloader
Redesigned user interface
Refactored just about everything

You can download it on the projects page.

Giving Credit Where Credit Is Due

Sun, 09 Jun 2013 01:51:00 +0000

A few months back, I wrote a [post](http://www.nateshoffner.com/2013/03/a-well-deserved-apology “A Well-Deserved Apology) where I apologized to Mathew Kemp (aka Sniped) from Jagex. I finally felt as though I had some closure. Some peace of mind. Everything changed when the Jagex nation attacked.

The other week, an interesting detail was brought to my attention. In a recent update of Ace of Spades, they decided to finally include some credits. Somebody showed me a screenshot of what appeared to be a smear campaign against the original developer. I figured it was Photoshopped, as no reputable company would target an individual like that.

I decided to check in-game for myself and what do you know, there it was.

"Even in the darkest days when one of our best men, the lead programmer, proved to be a double agent and attempted to leave the building with sensitive intel! Fear not the cops always get their man."

As a quick synopsis (as well as a bit of a preface) for those who don’t know, Ben Aksoy was the original developer of Ace of Spades. After selling the rights to Jagex, he later ended up working at their Cambridge office. In mid-late 2012, he was unhappy with the way things were being handled at the company and the direction in which the game was going, resulting in him putting in his final notice. After coming back from lunch one day, his hard drive in his personal laptop somehow magically broke, resulting in a BSOD. Additionally, stuff in his workspace was moved around. A fellow employee said the system admin had been at his desk. Upon approaching the sysadmin and asking about it, they denied everything.

This sounds a bit like a conspiracy at first and maybe it is. But the point remains that the hard drive just spontaneously broke over a short time period. Additionally, I don’t think anybody is surprised by some methods companies use in order to “protect” intellectual property and how poorly they treat their employees. For reference, feel free to read some of the reviews from current/previous Jagex employees online.

Before I propose my theory I have a bit of disclaimer. I’ll admit I have a bit of a personal bias against Jagex. On that same list is the Westboro Baptist Church, copyright/patent trolls, and whoever decided to cancel Oreo O’s. Basically, if you’re going to negatively impact/inhibit other people, society, or my morning ritual of pure bliss with milk, I’m going to have a personal bias against you.

Conspiracy #1

So this is my theory in tying the two incidents together. As soon as I read “attempted to leave the building with sensitive intel”, I immediately thought of that situation from a few months back. Perhaps Jagex was worried that Ben would be leaving with source code on his personal laptop. It seems like it’d be just tad extreme to destroy/swap a hard drive in order to protect an investment, but I wouldn’t rule it out entirely.

As for the “Fear not the cops always get their man” part, I’m not really sure what that entails. I haven’t heard anything regarding Ben and the authorities. I may be looking too far into it, but regardless that’s quite a bit of slander.

Conspiracy #2

Let’s move on and analyze those credits. First we need to take a solid look at that middle paragraph:

"Even in the darkest days when one of our best men, the lead programmer, proved to be a double agent and attempted to leave the building with sensitive intel! Fear not the cops always get their man. But as a team everyone gathered together and rallied and before you stands a game we are all tremendously proud of, enjoyed by a community we have incredible respect for. gathered together and rallied behind the game. Before you stands a game we are all tremendously proud of, enjoyed by a community we have incredible respect for."

Anybody with a 2nd grade education is probably scratching their heads right now. Look at the last 2 sentences and how they repeat. No that wasn’t my typo, that’s in the game. It looks as though somebody made a last minute edit and copy/pasted the text without any final proof-read. Perhaps the text was just shifted to the right during the edit. If that’s the case, I would like to know whether this is something that was known before release or if it was edited in at the very last minute without anybody noticing.

Conclusion

Regardless of conspiracies, motives and other drama, one point still remains. I don’t care what company you work for. No company should be personally attacking an individual like this. Even moreso after everything he was put through until he finally had enough. Is it not bad enough that you ripped the guy off and made false promises, only to screw him over in the end? Then you recreate a watered-down copy of the game that panders to drooling Call of Duty kids while completely alienating the existing userbase. The game has severely flopped and you have yet to make a real profit off of it. Yet your priorities still seem to be personally attacking the developer. Good game. (No, not Ace of Spades 1.0. Are you even paying attention?)

They made sure to give credit to Blitz Game Studios and Mat, but not the primary developer, at least not by name. It’s not as if they could’ve totally forgotten Ben’s involvement, which makes me think this was a known edit.

We could give Jagex the benefit of the doubt. Maybe they aren’t referring to Ben. Maybe it’s just complete satire which just happens to fit that exact situation perfectly. Maybe Jagex didn’t break his hard drive in his laptop before leaving. Maybe Jagex’s “little brother” snuck in and edited the credits without anybody knowing. Who knows?

Because I’m on a roll, here are some other points of interests within the credits:

"It has been a long and arduous campaign to get from a simple blocky green field to an international conflict involving hundreds of thousands of gamers."

Ahem, “hundreds of thousands” should be changed to “hundreds”. Don’t flatter yourself. You may have made claims of several hundred thousand users before, but the actual stats say otherwise.

"Without the awesome Ace of Spades community we would have not been anywhere near the success we have been."

Actually, if you would’ve listened to the community for once, maybe the game would’ve had a chance at actually being a success. Try checking your forums, steam community, in-game players, and the rest of the internet.

"To those of you who directly contributed to building Ace of Spades from the very early days; the code contributors, the community team, the mappers, the modders and the third party software creators, we recognise your efforts and want to give you your very own 21 gun salute!"

As somebody who contributed with the development of the game, the community, as well as my own software, you can keep your salute. You already desecrated one developer, leave myself and any others out of it.

And of course for reference, here are some screenshots of the credits:

This post in no way reflects the collective opinion of Build and Shoot and/or Buld Then Snip, LLC.

Update (06/17/2013)

As of July 13th (5 days after originally posting this entry), Jagex released an update for the game. The complete change log can be found [here](http://www.aceofspades.com/forums/showthread.php?20030-Update-Notes-13th-June-Discussion-Thread).

Not included in that change log is the fact that they completely removed the credits page. Instead, there is an image in the background crediting Blitz Game Studios.

So, kudos to Jagex for removing the defamatory messages from the credits, but they still failed to take responsibility for their mishap.

Not surprised that once again, you guys have have half-assed the job.

A Well-Deserved Apology

Sat, 02 Mar 2013 17:53:00 +0000

Months ago, amidst the chaos and commotion that is IRC, there were constant discussions with Jagex’s community managers Sniped (Mathew Kemp) and SallyTheButcher (Sally Da Costa). Conversation was often heated regarding the Ace of Spades community. This was primarily due to the Jagex community managers waving unwarranted sense of know-all in regards to the game and community based off “experience” from other games/communities. Most of which were dropped months after being launched, so they refused to properly cite said “experience”. Through all the bickering, I think it’s fair to say that nobody bumped heads more than Sniped and myself (IRC handle “StackOverflow”). This post is primarily directed at Sniped but occasionally Jagex as a whole.

September 2nd, 2012:

[16:12] <@StackOverflow> this is a waste of time, you can't talk to people who are too self-absorbed in their own ways of doing things 
[16:12] <sniped> well, you can go and do something else Stack
[16:13] <@StackOverflow> sniped, you're too clueless to have the slightest idea of what we're doing
[16:13] <sniped> kk stack, well come back in 6 months and I'll accept your apology

This quote is taken from a daily argument within an operator IRC channel (that we owned, mind you) and my statement was directed towards Build and Shoot, which at the time was being worked on privately.

Well, here we are. 6 months later. We’ve both changed a lot since then, haven’t we? I sure have. I’ve grown to the point where I can freely admit my wrongdoings and feel no particular shame, but rather learn from them.

So, lets get this going. I have a long, well deserved apology from myself to you:

I’m sorry that your incompetence and persistent self-assurance has resulted in you completely alienating yourselves from the community and in turn, the game entirely

You had a great opportunity to turn this game into something great. Now I don’t mean this in the typical bitter 12 year old fashion. I mean you quite literally had a flourishing community and found a way to piss them off in nearly everywhere possible, some things I would’ve never seen coming. For that, I applaud you.

I know instantly you would assume this is typical, biased banter. You seem to live in some weird alternate reality where Jagex does no wrong and you know the community well. With a game like Ace of Spades, community is everything. If you can’t handle the community, you are bound to fail. A vast majority of the heated flaming towards Jagex was (and still is) at least mildy constructive and yet you decide to focus on the unconstructive criticisms from 12 year olds. I’m not sure if you do this intentionally or you are just unable to differentiate between blatant flaming/trolling and constructive criticism. I’m assuming the latter since this was an ongoing problem on IRC, where you and Sally would focus on people spamming instead of people with legitimate questions (myself included).

Danhezee and myself were often subject to blame for the community’s negative sentiment towards Jagex. Early on, that might have seemed like the case, sure, assuming people aren’t able to form an opinion on their own. It’s very easy to blame a handful of people instead of taking responsibility for your own mismanagement. However, it’s gotten to the point where the criticism has moved beyond just the community. For example, Gamasutra has written an article titled “How Ace of Spades gave Jagex a lesson in community management”. All over the internet you see people complaining about the way Jagex failed to handle the community. I’m curious as to whether you are still so disillusioned as to think that this is the handy work of a few disgruntled individuals or if you’ve finally realized that you’re to blame.

I’m sorry that your marketing campaigns have backfired

Early on, it was quite obvious you were a fan of numbers, saying things like “well, when we have 3 million players…” when discussing scalability, obsessing over Facebook “likes”, along with telling the community there were “over 500k” pre-registered players for 1.0. Scalability is great, but so is realism. At the time, the beta was holding a steady 3,800 players. This was without any sort of advertising and without updates for months on end.

Before release, there was a massive marketing campaign involving paying YouTubers (even though you try to deny it, it's obvious),  media outlets, and even "major" news outlets like [NBC](http://www.nbcnews.com/technology/ingame/machine-guns-meet-minecraft-ace-spades-1C6814146). (Because we all know NBC had nothing better to report than the release of Ace of Spades 1.0).

Promotional graphics were obviously rendered, which wouldn’t have been so much of an issue had you not openly stated they were in-game screenshots. Spoiler: They weren’t.

Now, in retrospect, 1.0 can barely muster more than 1,500 players at any given time with the exception of when it is 50% off on Steam. Any game will sell well on Steam if it’s 50% off. Look a graph charting the playercounts over the last few months and you can easily determine when the game was on sale. 2-3 days after a steam sale, players drop right down to where they were beforehand.

There is nothing that is captivating them. Sure, you got their money, but that’s not sustainable in the long term. Let’s do some math here. I’ve heard Valve takes roughly a 30% cut of sales, I’m not sure how accurate that is myself, if anybody knows for certain feel free to let me know, but it feels like a safe estimate. Based off that estimate, for every 1,000 copies you sell while it is 50% off, Jagex is only making $3,500. Keep in mind, they have been dumping money into this game since early-mid 2011. There have been massive advertising efforts, developer teams (both internal and external), third-party code payoffs (pyspades), servers, and more spanned over multiple years. This is in no way sustainable.

Now there is paid DLC for sale, which is a whole discussion in itself. Even after saying there would be no paid DLC, I think they might be starting to realize that they aren’t going to break even. Next, they’ll likely start leasing privatized servers in an effort to cover server costs. Even though the game servers are hosted on cheap VPSs, it still adds up. People want the flexibility of open server software, not just locked down servers. Sadly, Jagex will still just look at this as another method for potential profit and half-ass it like always.

Some of the primary factors in Ace of Spades’ marketing seem to be:

Misleading figures
Misleading promotional content (graphics, maps, etc)
Misleading system specs. (The steam page claimed it required DirectX 9.0 for the longest time when the game uses OpenGL)

Have fun explaining those figures to your venture capitalists. (As a side note: Anybody who says you should buy 1.0 to “support” the developers, clearly has no idea how Jagex is financially structured.)

I’m sorry that your lack of technical prowess has come back to bite you in the ass

This was evidently a touchy subject, seeing as I was banned on the forums shortly after posting it on the forums.

After seeing you write this:

Sniped wrote: The botting issue in the prototype of AoS will not exist when it is fully released. Simple as that.

I replied with the statement below, resulting in me being banned for “flaming other users”.

lol. Please learn how aimbots work before making a statement like this.

This reminds me of a quote off of IRC:

<Sniped> if there is no cheating possible, would you need extra admin tools?

You shouldn’t make such bold implications, especially if they have no technical backing.

People will cheat in 1.0 just as with any game. OpenGL as a technology is tried and tested when it comes to cheating.

I wouldn’t consider idle legal threats as being a “wealth of experience”. I’ve done my fair share of anti-cheat measures, both technically and socially. I have real-world knowledge and real-world expectations about the subject. Blindly expecting Jagex to obliterate all cheating is quite preposterous. Especially when trying to pull a line like “We have a wealth of experience in dealing with bots too.”, when everybody knows Jagex has little to no control over the cheating in their headlining game.

Protip: The average 12 year old who downloads an aimbot isn’t going to care about some idle legal threat. It’s quite laughable.

I speak for myself when saying I don’t take a company seriously if they try to threaten a lawsuit against anybody who cheats in a game. If anything, it’s likely to stir up interest in people to make cheats as a sort of challenge.

Also, mfw everybody is pretty much reiterating what I wrote in a post, only to have it deleted for being “libelous”. The only problem is my statement was based of of personal expereience from RuneScape, as well as public documentation regarding legalites between Jagex and cheaters. Basically, it’s proven and factual (although my personal experence with RS would be considered opinionated). That’s not anywhere near being “libelous”.

This is just one of many examples where your lack of real-world experience/knowledge has come back to haunt you. By making bold statements and implications such as those, you ignited the interest in many individuals to create cheats, some of which were out on launch day. Why? I’ll give you a hint, people will always cheat. People will do it for whatever reason they want. But for those who make cheats, it’s generally for the challenge to do so and when you antagonize people with statements such as those, you’re only adding fuel to the fire. So congratulations. You not only proved how ignorant you are, but you also helped ruin the in-game experience for players.

Here’s a little bit of advice. I know at Jagex, you guys are quote fond of extravagant (and rather pretentious) titles, but those titles are meaningless unless you have real knowledge, and more importantly, real-world experience. A popular trend among Jagex employees seems to be that of praising the “specialists” within the company. Any problem you can think of, there’s evidently a specialist for it. I’m not doubting that there are people who could know what they’re doing there, but if YOU don’t have an accurate estimation of their skillset, I’m not going to take your word for it. Not to mention, there are many track records with how Jagex has handled, or failed to handle certain situations. Take cheating for instance. You claim you have “anti-cheat specialists”. Where are they? I’m going to assume they’re not involved with Ace of Spades, otherwise you wouldn’t leave such gaping holes like leaving a god damn console within the game.

I’m sorry for your skewed outlook of Build and Shoot and its denial of partnership

The Ace of Spades community as a whole was happy to see Build and Shoot carrying on support for the free to play version and the response to this by Jagex was censorship and bans. I could partially understand this from a company perspective. If people know there’s a free version, they’ll be less inclined to buy the paid version. However, it’s become quite apparent to everybody that the main reason for the irrational responses were the result of petty differences still lingering between the old Ace of Spades staff and Jagex’s community managers.

As a result, the only people who were negatively effected by this was the community, not Build and Shoot. On the contrary, the constant censorship resulted in a Streisand effect and it became even more popular, so thank you for that.

Build and Shoot extended its arms with Jagex in an attempt to patch things up. We tried appeasing Jagex by adding sections on our forums for the 1.0 release. After an exchange of emails, primarily with Sniped, we realized there was always a blockade between us and their appeasement. A snippet from the last email reads:

The passion of your community has overflowed onto some of our other platforms, such as social media and Steam. We have mentioned this to you before but have yet to see any attempts by the site administrators to deal with this issue. This issue is not something which just affects us as the behavior of some of these people who represent your community is going to reflect badly on you.

As I mentioned earlier, you fail to take responsibility for your own mistakes and are under the severe delusion that we are responsible for the negative sentiment towards Jagex.

Partnering with Build and Shoot would’ve only resulted positively for Jagex, yet petty, childish, disagreements still seem to be a problem. The community knows about Build and Shoot, so there’s no real point in censoring it. Partnership would’ve meant less work on your end in terms of moderation and would’ve helped appease the community greatly. It could’ve helped serve as a marketing advantage for new players who are introduced to Build and Shoot and could possibly become interested in 1.0, sadly you seemed to look at things through a very narrow scope.

I’ll cut it short with the apologies. I’m sure there is more to say, but I feel all 3 people who have read this far want to stab me with a soldering iron for wasting their time. Thankfully I actually remembered to type this up today, as I wouldn’t want my punctual reputation to be blemished. As I said, the game has always been centered around the community. Your failure to properly manage it has resulted in the entire game spiraling downward. Sorry you had to learn that the hard way.

Amidst all of these apologies, I must grant you very sincere “thank you” for helping show me where having an undeserved sense of accomplishment and self-righteous attitude can get you. Second, I would like to thank you for helping make Build and Shoot what it is now and for it’s continuous growth. We have a thriving community that is completely decentralized. Any attempts at patching things up with Jagex has been mainly a philanthropic effort for the community. Granted we would get publicity, which is nice. But you have given us more than enough through your failed attempts at censoring us and the community.

A while back, I was offered to work at Jagex in Cambridge to help develop Ace of Spades. At the time it seemed like a fantastic opportunity. Sadly, I declined the offer, so we didn’t get to share this roller coaster experience as drinking buddies. But judging by your actions, incoherent thinking, and constantly tripping over yourself, you’ve been drunk this entire time.

tl;dr Sorry.

Sincerely, Nate Shoffner, Senior apology specialist.

This post in no way reflects the collective opinion of Build and Shoot and/or Buld Then Snip, LLC.

Update (3/5/2013):

It seems this article has been passed around quite a bit throughout Jagex.

It has come to my attention that Jagex has been censoring this article on their platforms. Now this doesn’t come as a surprise due to the nature of the article, but their reason for doing so is a bit absurd, in my opinion. Initially, the reasoning for deleting the posts was because it mentioned Build and Shoot. Sorry that Build and Shoot’s success is starting to overshadow your own.

One of the forum staff, lods, said this:

“it’s directing people to a non constructive article which is full of inaccuracies…”

This might be a biased opinion, but I’d say that the article was at least mildy constructive. As for “inaccuracies”, unless you can provide direct examples, don’t say anything. Maybe instead of just skimming the article and making a baseless assumption, you could actually read it before claiming there are “inaccuracies”.

Jagex’s response to this article has done nothing but prove the very behavior it was criticizing.

Thanks for proving my point, again.

C# - Reusable Portable Application Settings

Thu, 24 Jan 2013 05:19:00 +0000

Anybody who has worked with the .NET Framework has likely dealt with the native configuration files, especially if you’re using something as intuitive as Visual Studio. While the native functionality is pretty nifty, there’s still one small gripe myself and many other developers have. The .NET Framework is designed in a way that applications are to interface with a single configuration file whose location is found somewhere between AppData and obscurity. The reason for this design, according to Microsoft, was to alleviate the possibility of overwrite collisions between different applications. Despite being asked to allow developers to manage their own relative configuration paths, Microsoft has stood by this design. It’s quite an annoying “feature”, but there are workarounds, albeit tedious. This article provides a good bit of insight on how to tackle something like this. by inheriting the SettingsProvider class.

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Configuration;
using System.Linq;
using System.Text;
using Microsoft.Win32;
using System.Xml;
using System.Xml.Serialization;
 
public class PortableSettingsProvider : SettingsProvider
{
    // Define some static strings later used in our XML creation
    // XML Root node
    const string XMLROOT = "configuration";
 
    // Configuration declaration node
    const string CONFIGNODE = "configSections";
 
    // Configuration section group declaration node
    const string GROUPNODE = "sectionGroup";
 
    // User section node
    const string USERNODE = "userSettings";
 
    // Application Specific Node
    string APPNODE = System.Reflection.Assembly.GetExecutingAssembly().GetName().Name + ".Properties.Settings";
 
    private System.Xml.XmlDocument xmlDoc = null;
 
 
 
    // Override the Initialize method
    public override void Initialize(string name, NameValueCollection config)
    {
        base.Initialize(this.ApplicationName, config);
    }
 
    // Override the ApplicationName property, returning the solution name.  No need to set anything, we just need to
    // retrieve information, though the set method still needs to be defined.
    public override string ApplicationName
    {
        get
        {
            return (System.Reflection.Assembly.GetExecutingAssembly().GetName().Name);
        }
        set
        {
            return;
        }
    }
 
    // Simply returns the name of the settings file, which is the solution name plus ".config"
    public virtual string GetSettingsFilename()
    {
        return ApplicationName + ".config";
    }
 
    // Gets current executable path in order to determine where to read and write the config file
    public virtual string GetAppPath()
    {
        return new System.IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly().Location).DirectoryName;
    }
 
    // Retrieve settings from the configuration file
    public override SettingsPropertyValueCollection GetPropertyValues(SettingsContext sContext, SettingsPropertyCollection settingsColl)
    {
        // Create a collection of values to return
        SettingsPropertyValueCollection retValues = new SettingsPropertyValueCollection();
 
        // Create a temporary SettingsPropertyValue to reuse
        SettingsPropertyValue setVal;
 
        // Loop through the list of settings that the application has requested and add them
        // to our collection of return values.
        foreach (SettingsProperty sProp in settingsColl)
        {
            setVal = new SettingsPropertyValue(sProp);
            setVal.IsDirty = false;
            setVal.SerializedValue = GetSetting(sProp);
            retValues.Add(setVal);
        }
        return retValues;
    }
     
    // Save any of the applications settings that have changed (flagged as "dirty")
    public override void SetPropertyValues(SettingsContext sContext, SettingsPropertyValueCollection settingsColl)
    {
        // Set the values in XML
        foreach (SettingsPropertyValue spVal in settingsColl)
        {
            SetSetting(spVal);
        }
 
        // Write the XML file to disk
        try
        {
            XMLConfig.Save(System.IO.Path.Combine(GetAppPath(), GetSettingsFilename()));
        }
        catch (Exception ex)
        {
            // Create an informational message for the user if we cannot save the settings.
            // Enable whichever applies to your application type.
             
            // Uncomment the following line to enable a MessageBox for forms-based apps
            //System.Windows.Forms.MessageBox.Show(ex.Message, "Error writting configuration file to disk", System.Windows.Forms.MessageBoxButtons.OK, System.Windows.Forms.MessageBoxIcon.Error);
 
            // Uncomment the following line to enable a console message for console-based apps
            //Console.WriteLine("Error writing configuration file to disk: " + ex.Message);
        }
    }
 
    private XmlDocument XMLConfig
    {
        get
        {
            // Check if we already have accessed the XML config file. If the xmlDoc object is empty, we have not.
            if (xmlDoc == null)
            {
                xmlDoc = new XmlDocument();
 
                // If we have not loaded the config, try reading the file from disk.
                try
                {
                    xmlDoc.Load(System.IO.Path.Combine(GetAppPath(), GetSettingsFilename()));
                }
 
                // If the file does not exist on disk, catch the exception then create the XML template for the file.
                catch (Exception)
                {
                    // XML Declaration
                    // <?xml version="1.0" encoding="utf-8"?>
                    XmlDeclaration dec = xmlDoc.CreateXmlDeclaration("1.0", "utf-8", null);
                    xmlDoc.AppendChild(dec);
 
                    // Create root node and append to the document
                    // <configuration>
                    XmlElement rootNode = xmlDoc.CreateElement(XMLROOT);
                    xmlDoc.AppendChild(rootNode);
 
                    // Create Configuration Sections node and add as the first node under the root
                    // <configSections>
                    XmlElement configNode = xmlDoc.CreateElement(CONFIGNODE);
                    xmlDoc.DocumentElement.PrependChild(configNode);
 
                    // Create the user settings section group declaration and append to the config node above
                    // <sectionGroup name="userSettings"...>
                    XmlElement groupNode = xmlDoc.CreateElement(GROUPNODE);
                    groupNode.SetAttribute("name", USERNODE);
                    groupNode.SetAttribute("type", "System.Configuration.UserSettingsGroup");
                    configNode.AppendChild(groupNode);
 
                    // Create the Application section declaration and append to the groupNode above
                    // <section name="AppName.Properties.Settings"...>
                    XmlElement newSection = xmlDoc.CreateElement("section");
                    newSection.SetAttribute("name", APPNODE);
                    newSection.SetAttribute("type", "System.Configuration.ClientSettingsSection");
                    groupNode.AppendChild(newSection);
 
                    // Create the userSettings node and append to the root node
                    // <userSettings>
                    XmlElement userNode = xmlDoc.CreateElement(USERNODE);
                    xmlDoc.DocumentElement.AppendChild(userNode);
 
                    // Create the Application settings node and append to the userNode above
                    // <AppName.Properties.Settings>
                    XmlElement appNode = xmlDoc.CreateElement(APPNODE);
                    userNode.AppendChild(appNode);
                }
            }
            return xmlDoc;
        }
    }
 
    // Retrieve values from the configuration file, or if the setting does not exist in the file,
    // retrieve the value from the application's default configuration
    private object GetSetting(SettingsProperty setProp)
    {
        object retVal;
        try
        {
            // Search for the specific settings node we are looking for in the configuration file.
            // If it exists, return the InnerText or InnerXML of its first child node, depending on the setting type.
 
            // If the setting is serialized as a string, return the text stored in the config
            if (setProp.SerializeAs.ToString() == "String")
            {
                return XMLConfig.SelectSingleNode("//setting[@name='" + setProp.Name + "']").FirstChild.InnerText;
            }
 
            // If the setting is stored as XML, deserialize it and return the proper object.  This only supports
            // StringCollections at the moment - I will likely add other types as I use them in applications.
            else
            {
                string settingType = setProp.PropertyType.ToString();
                string xmlData = XMLConfig.SelectSingleNode("//setting[@name='" + setProp.Name + "']").FirstChild.InnerXml;
                XmlSerializer xs = new XmlSerializer(typeof(string[]));
                string[] data = (string[])xs.Deserialize(new XmlTextReader(xmlData, XmlNodeType.Element, null));
 
                switch (settingType)
                {
                    case "System.Collections.Specialized.StringCollection":
                        StringCollection sc = new StringCollection();
                        sc.AddRange(data);
                        return sc;
                    default:
                        return "";
                }
            }
        }
        catch (Exception)
        {
            // Check to see if a default value is defined by the application.
            // If so, return that value, using the same rules for settings stored as Strings and XML as above
            if ((setProp.DefaultValue != null))
            {
                if (setProp.SerializeAs.ToString() == "String")
                {
                    retVal = setProp.DefaultValue.ToString();
                }
                else
                {
                    string settingType = setProp.PropertyType.ToString();
                    string xmlData = setProp.DefaultValue.ToString();
                    XmlSerializer xs = new XmlSerializer(typeof(string[]));
                    string[] data = (string[])xs.Deserialize(new XmlTextReader(xmlData, XmlNodeType.Element, null));
 
                    switch (settingType)
                    {
                        case "System.Collections.Specialized.StringCollection":
                            StringCollection sc = new StringCollection();
                            sc.AddRange(data);
                            return sc;
 
                        default: return "";
                    }
                }
            }
            else
            {
                retVal = "";
            }
        }
        return retVal;
    }
 
    private void SetSetting(SettingsPropertyValue setProp)
    {
        // Define the XML path under which we want to write our settings if they do not already exist
        XmlNode SettingNode = null;
 
        try
        {
            // Search for the specific settings node we want to update.
            // If it exists, return its first child node, (the <value>data here</value> node)
            SettingNode = XMLConfig.SelectSingleNode("//setting[@name='" + setProp.Name + "']").FirstChild;
        }
        catch (Exception)
        {
            SettingNode = null;
        }
 
        // If we have a pointer to an actual XML node, update the value stored there
        if ((SettingNode != null))
        {
            if (setProp.Property.SerializeAs.ToString() == "String")
            {
                SettingNode.InnerText = setProp.SerializedValue.ToString();
            }
            else
            {
                // Write the object to the config serialized as Xml - we must remove the Xml declaration when writing
                // the value, otherwise .Net's configuration system complains about the additional declaration.
                SettingNode.InnerXml = setProp.SerializedValue.ToString().Replace(@"<?xml version=""1.0"" encoding=""utf-16""?>", "");
            }
        }
        else
        {
            // If the value did not already exist in this settings file, create a new entry for this setting
 
            // Search for the application settings node (<Appname.Properties.Settings>) and store it.
            XmlNode tmpNode = XMLConfig.SelectSingleNode("//" + APPNODE);
 
            // Create a new settings node and assign its name as well as how it will be serialized
            XmlElement newSetting = xmlDoc.CreateElement("setting");
            newSetting.SetAttribute("name", setProp.Name);
             
            if (setProp.Property.SerializeAs.ToString() == "String")
            {
                newSetting.SetAttribute("serializeAs", "String");
            }
            else
            {
                newSetting.SetAttribute("serializeAs", "Xml");
            }
 
            // Append this node to the application settings node (<Appname.Properties.Settings>)
            tmpNode.AppendChild(newSetting);
 
            // Create an element under our named settings node, and assign it the value we are trying to save
            XmlElement valueElement = xmlDoc.CreateElement("value");
            if (setProp.Property.SerializeAs.ToString() == "String")
            {
                valueElement.InnerText = setProp.SerializedValue.ToString();
            }
            else
            {
                // Write the object to the config serialized as Xml - we must remove the Xml declaration when writing
                // the value, otherwise .Net's configuration system complains about the additional declaration
                valueElement.InnerXml = setProp.SerializedValue.ToString().Replace(@"<?xml version=""1.0"" encoding=""utf-16""?>", "");
            }           
 
            //Append this new element under the setting node we created above
            newSetting.AppendChild(valueElement);
        }
    }
}

Deserialization Bug

Immediately I found a bug during deserialization using ArrayList or StringCollection within GetSetting(). It seems that the collection was being cast to a string array during deserialization. Luckily this is a simple fix:

private object GetSetting(SettingsProperty setProp)
{
    object retVal;
     
    try
    {
        if (setProp.SerializeAs.ToString() == "String")
        {
            return XMLConfig.SelectSingleNode("//setting[@name='" + setProp.Name + "']").FirstChild.InnerText;
        }
 
        var xmlData = XMLConfig.SelectSingleNode(string.Format("//setting[@name='{0}']", setProp.Name)).FirstChild.InnerXml;
        return string.Format(@"{0}", xmlData);
    }
     
    catch (Exception)
    {
        if ((setProp.DefaultValue != null))
        {
            if (setProp.SerializeAs.ToString() == "String")
            {
                retVal = setProp.DefaultValue.ToString();
            }
            else
            {
                var settingType = setProp.PropertyType.ToString();
                var xmlData = setProp.DefaultValue.ToString();
                var xs = new XmlSerializer(typeof (string[]));
                var data = (string[]) xs.Deserialize(new XmlTextReader(xmlData, XmlNodeType.Element, null));
 
                switch (settingType)
                {
                    case "System.Collections.Specialized.StringCollection":
                        var sc = new StringCollection();
                        sc.AddRange(data);
                        return sc;
 
                    default:
                        return "";
                }
            }
        }
        else
        {
            retVal = "";
        }
    }
    return retVal;
}

Making it Reusable

Great, now it works perfectly. However, I’m far too lazy to copy this snippet into every project, I’d much prefer to reuse it from a referenced library. In order to do this, you’ll have to make use of GetEntryAssembly(). This will allow you to reference the portable settings class within any assembly. Some words of caution: This solution is dependent on the calling assembly name. It will create a directory in the ApplicationData folder using the Environment.GetFolderPath() method and then create the appropriate configuration file using the same name. Use it at your own risk and with common sense, especially if you plan on using this in any sort of production software.

Create a standard library (DLL) and add the following class:

public class PortableSettingsProvider : SettingsProvider
{
    private const string XMLROOT = "configuration"; // XML Root node
    private const string CONFIGNODE = "configSections"; // Configuration declaration node     
    private const string GROUPNODE = "sectionGroup"; // Configuration section group declaration node
    private const string USERNODE = "userSettings"; // User section node
 
    // Application Specific Node
 
    private static string ConfigName;
    private static string ConfigDirectory;
    private string APPNODE;
 
    // Store instace of calling assembly
    private Assembly entryAssembly;
 
    private XmlDocument xmlDoc;
 
    public override string ApplicationName
    {
        get { return (entryAssembly.GetName().Name); }
        set { APPNODE = value; }
    }
 
    private XmlDocument XMLConfig
    {
        get
        {
            // Check if we already have accessed the XML config file. If the xmlDoc object is empty, we have not.
            if (xmlDoc == null)
            {
                xmlDoc = new XmlDocument();
 
                // If we have not loaded the config, try reading the file from disk.
                try
                {
                    xmlDoc.Load(Path.Combine(GetAppPath(), GetSettingsFilename()));
                }
 
                    // If the file does not exist on disk, catch the exception then create the XML template for the file.
                catch (Exception)
                {
                    // XML Declaration
                    // <?xml version="1.0" encoding="utf-8"?>
                    var dec = xmlDoc.CreateXmlDeclaration("1.0", "utf-8", null);
                    xmlDoc.AppendChild(dec);
 
                    // Create root node and append to the document
                    // <configuration>
                    var rootNode = xmlDoc.CreateElement(XMLROOT);
                    xmlDoc.AppendChild(rootNode);
 
                    // Create Configuration Sections node and add as the first node under the root
                    // <configSections>
                    var configNode = xmlDoc.CreateElement(CONFIGNODE);
                    xmlDoc.DocumentElement.PrependChild(configNode);
 
                    // Create the user settings section group declaration and append to the config node above
                    // <sectionGroup name="userSettings"...>
                    var groupNode = xmlDoc.CreateElement(GROUPNODE);
                    groupNode.SetAttribute("name", USERNODE);
                    groupNode.SetAttribute("type", "System.Configuration.UserSettingsGroup");
                    configNode.AppendChild(groupNode);
 
                    // Create the Application section declaration and append to the groupNode above
                    // <section name="AppName.Properties.Settings"...>
                    var newSection = xmlDoc.CreateElement("section");
                    newSection.SetAttribute("name", APPNODE);
                    newSection.SetAttribute("type", "System.Configuration.ClientSettingsSection");
                    groupNode.AppendChild(newSection);
 
                    // Create the userSettings node and append to the root node
                    // <userSettings>
                    var userNode = xmlDoc.CreateElement(USERNODE);
                    xmlDoc.DocumentElement.AppendChild(userNode);
 
                    // Create the Application settings node and append to the userNode above
                    // <AppName.Properties.Settings>
                    var appNode = xmlDoc.CreateElement(APPNODE);
                    userNode.AppendChild(appNode);
                }
            }
            return xmlDoc;
        }
    }
 
    // Override the Initialize method
    public override void Initialize(string name, NameValueCollection config)
    {
        entryAssembly = Assembly.GetEntryAssembly();
 
        ConfigName = entryAssembly.GetName().Name;
 
        ConfigDirectory = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), entryAssembly.GetName().Name);
 
        if (!(Directory.Exists(ConfigDirectory)))
        {
            Directory.CreateDirectory(ConfigDirectory);
        }
 
        APPNODE = ConfigName + ".Properties.Settings";
 
 
        base.Initialize(ApplicationName, config);
    }
 
    // Simply returns the name of the settings file, which is the solution name plus ".config"
    public virtual string GetSettingsFilename()
    {
        return string.Format("{0}.config", ApplicationName);
    }
 
    // Gets current executable path in order to determine where to read and write the config file
    public virtual string GetAppPath()
    {
        return ConfigDirectory;
        //return new FileInfo(callingAssembly.Location).DirectoryName;
    }
 
    // Override the ApplicationName property, returning the solution name.  No need to set anything, we just need to
    // retrieve information, though the set method still needs to be defined.
 
 
    // Retrieve settings from the configuration file
    public override SettingsPropertyValueCollection GetPropertyValues(SettingsContext sContext, SettingsPropertyCollection settingsColl)
    {
        // Create a collection of values to return
        var retValues = new SettingsPropertyValueCollection();
 
        // Create a temporary SettingsPropertyValue to reuse
 
        // Loop through the list of settings that the application has requested and add them
        // to our collection of return values.
        foreach (SettingsProperty sProp in settingsColl)
        {
            var setVal = new SettingsPropertyValue(sProp) {IsDirty = false, SerializedValue = GetSetting(sProp)};
            retValues.Add(setVal);
        }
        return retValues;
    }
 
    // Save any of the applications settings that have changed (flagged as "dirty")
    public override void SetPropertyValues(SettingsContext sContext, SettingsPropertyValueCollection settingsColl)
    {
        // Set the values in XML
        foreach (SettingsPropertyValue spVal in settingsColl)
        {
            SetSetting(spVal);
        }
 
        // Write the XML file to disk
        try
        {
            XMLConfig.Save(Path.Combine(GetAppPath(), GetSettingsFilename()));
        }
        catch (Exception ex)
        {
            // Create an informational message for the user if we cannot save the settings.
            // Enable whichever applies to your application type.
 
            // Uncomment the following line to enable a MessageBox for forms-based apps
            //MessageBox.Show(ex.Message, "Error writting configuration file to disk", MessageBoxButtons.OK, MessageBoxIcon.Error);
 
            // Uncomment the following line to enable a console message for console-based apps
            //Console.WriteLine("Error writing configuration file to disk: " + ex.Message);
        }
    }
 
    // Retrieve values from the configuration file, or if the setting does not exist in the file,
    // retrieve the value from the application's default configuration
    private object GetSetting(SettingsProperty setProp)
    {
        object retVal;
        try
        {
            // Search for the specific settings node we are looking for in the configuration file.
            // If it exists, return the InnerText or InnerXML of its first child node, depending on the setting type.
 
            // If the setting is serialized as a string, return the text stored in the config
            if (setProp.SerializeAs.ToString() == "String")
            {
                return XMLConfig.SelectSingleNode("//setting[@name='" + setProp.Name + "']").FirstChild.InnerText;
            }
 
            // This solves the problem with StringCollections throwing a NullReferenceException
            var xmlData = XMLConfig.SelectSingleNode(string.Format("//setting[@name='{0}']", setProp.Name)).FirstChild.InnerXml;
            return string.Format(@"{0}", xmlData);
        }
        catch (Exception)
        {
            // Check to see if a default value is defined by the application.
            // If so, return that value, using the same rules for settings stored as Strings and XML as above
            if ((setProp.DefaultValue != null))
            {
                if (setProp.SerializeAs.ToString() == "String")
                {
                    retVal = setProp.DefaultValue.ToString();
                }
                else
                {
                    var settingType = setProp.PropertyType.ToString();
                    var xmlData = setProp.DefaultValue.ToString();
                    var xs = new XmlSerializer(typeof (string[]));
                    var data = (string[]) xs.Deserialize(new XmlTextReader(xmlData, XmlNodeType.Element, null));
 
                    switch (settingType)
                    {
                        case "System.Collections.Specialized.StringCollection":
                            var sc = new StringCollection();
                            sc.AddRange(data);
                            return sc;
 
                        default:
                            return "";
                    }
                }
            }
            else
            {
                retVal = "";
            }
        }
        return retVal;
    }
 
    private void SetSetting(SettingsPropertyValue setProp)
    {
        // Define the XML path under which we want to write our settings if they do not already exist
        XmlNode SettingNode;
 
        try
        {
            // Search for the specific settings node we want to update.
            // If it exists, return its first child node, (the <value>data here</value> node)
            SettingNode = XMLConfig.SelectSingleNode(string.Format("//setting[@name='{0}']", setProp.Name)).FirstChild;
        }
        catch (Exception)
        {
            SettingNode = null;
        }
 
        // If we have a pointer to an actual XML node, update the value stored there
        if ((SettingNode != null))
        {
            if (setProp.Property.SerializeAs.ToString() == "String")
            {
                SettingNode.InnerText = setProp.SerializedValue.ToString();
            }
            else
            {
                // Write the object to the config serialized as Xml - we must remove the Xml declaration when writing
                // the value, otherwise .Net's configuration system complains about the additional declaration.
                SettingNode.InnerXml = setProp.SerializedValue.ToString().Replace(@"<?xml version=""1.0"" encoding=""utf-16""?>", "");
            }
        }
        else
        {
            // If the value did not already exist in this settings file, create a new entry for this setting
 
            // Search for the application settings node (<Appname.Properties.Settings>) and store it.
            var tmpNode = XMLConfig.SelectSingleNode(string.Format("//{0}", APPNODE)) ?? XMLConfig.SelectSingleNode(string.Format("//{0}.Properties.Settings", APPNODE));
 
            // Create a new settings node and assign its name as well as how it will be serialized
            var newSetting = xmlDoc.CreateElement("setting");
            newSetting.SetAttribute("name", setProp.Name);
            newSetting.SetAttribute("serializeAs", setProp.Property.SerializeAs.ToString() == "String" ? "String" : "Xml");
 
            // Append this node to the application settings node (<Appname.Properties.Settings>)
            tmpNode.AppendChild(newSetting);
 
            // Create an element under our named settings node, and assign it the value we are trying to save
            var valueElement = xmlDoc.CreateElement("value");
            if (setProp.Property.SerializeAs.ToString() == "String")
            {
                valueElement.InnerText = setProp.SerializedValue.ToString();
            }
            else
            {
                // Write the object to the config serialized as Xml - we must remove the Xml declaration when writing
                // the value, otherwise .Net's configuration system complains about the additional declaration
                valueElement.InnerXml = setProp.SerializedValue.ToString().Replace(@"<?xml version=""1.0"" encoding=""utf-16""?>", "");
            }
 
            //Append this new element under the setting node we created above
            newSetting.AppendChild(valueElement);
        }
    }
}

Assigning the ProviderBase

Next, open your Visual Studio solution and go to the Settings tab located in the project properties for the appropriate project. For each of the settings that you want to make portable, modify the “Provider” property to represent the namespace scheme for your portable settings class. You will have to modify the provider property for each subsequent setting if you want it to remain portable. It can be a bit of a hassle, but in the end, it’s a small cost for better control.

I have been using this code for a few years now on a few smaller projects with no problems. As I’ve stated, it’s not perfect, but it’s still an alright solution, assuming you use it properly.

Elasticon 2024

RRTA ModeShift Reverse-Engineering

LCWC and Dispatch Rabbit Holes

Method #1 - Web Scraping

Method #2 - RSS Feed

Method #3 - ESRI ArcGIS

Remaining Issues

Future Plans

Closing Thoughts

Google Play Developer Account Termination

Disable Nvidia Telemetry 1.1

Download

Intel Core i7-7700K Delidding

What is Delidding?

Build Specs

Benchmarks and Stress Tests:

Delidding Materials:

Conclusion and Disclaimer

Disable Nvidia Telemetry

Seachem Doser 1.2

Tablet:

Phone:

Moved to GitHub Pages & Site Redesign

Seachem Doser 1.1

Tablet

Phone:

Dropbox Does Not Access Files Throughout Your PC

"Pretend You're Xyzzy" Server Setup and Build Instructions

Programming - Paradigms, Satire, and Precedents

Introducing C+=

Update 12/15/2013

Update 12/19/2013

Programming as an Essential Skill Set

The real value of programming

Spadille 1.6 - Better Late Than Never?

Related Notes:

C# - Global Hotkeys via GetAsyncKeyState

Modifier Keys

Hooking & Unhooking

Prioritization

Polling

Tabster 1.6 Released

Giving Credit Where Credit Is Due

Conspiracy #1

Conspiracy #2

Conclusion

Update (06/17/2013)

A Well-Deserved Apology

I’m sorry that your incompetence and persistent self-assurance has resulted in you completely alienating yourselves from the community and in turn, the game entirely

I’m sorry that your marketing campaigns have backfired

I’m sorry that your lack of technical prowess has come back to bite you in the ass

I’m sorry for your skewed outlook of Build and Shoot and its denial of partnership

Update (3/5/2013):

C# - Reusable Portable Application Settings

Deserialization Bug

Making it Reusable

Assigning the ProviderBase