Two winters ago I left a position as a system administrator that was paying pretty well and moved cross-country to a region with less jobs than where I moved from. Three months later, I was still unemployed, broke, and bored. I was talking to my good friend Japhy on IRC one day and he was explaining to me how the tf-idf algorithm works. For reasons involving boredom more than any other reason, I dreamed up an idea: I would write software that would take a given document and generate book suggestions based on its content.

I think that most programmers would agree with me that we put in longer hours on code when we’re not working for anybody. We don’t stop learning, either. To us, unemployment is a brief sprint of academia spent in our home office, the local coffee shop, or our parent’s house. My imagination dreamed up this fairly straightforward process:

1. Take a given document and calculate tf-idf scores on all terms
2. Select X number of the highest scoring terms
3. Pass these high-scoring terms to an Amazon ItemSearch query
4. Receive a list of recommended books (with URLs) from Amazon

I had already written multiple Twitter bots by this time so I decided to just use some of my existing code to poll Twitter’s search API. Essentially, the “documents” I mentioned above were actually tweets containing the terms “book” or “books.” Two and a half days later I had a working prototype that could generate a book recommendation from a given tweet. It was at this time that I added steps 5 and 6:

Tag URLs returned from Amazon’s ItemSearch with an affiliate ID; and
Reply to the tweeting user with their new book suggestion

Four months later and I had generated over $7,000 in sales for Amazon with over $400 commission for myself. Obviously, the commission I was making wasn’t livable but it was a nice addition to my then-depleting savings. Had I decided to scale out my operation, I could have made much more. My benchmark is at four months because that’s how long I went before being suspended. My conversion rate? 0.13%! While seemingly low, this number is very high when compared to email spam. However, it’s important to note that email spam is subject to various filtering technologies. A fair amount of the time I share this story, people are more impressed with the fact that I went 4 months before getting suspended. The truth is, I had a lot of throttling built into my spam bot. The factors I think are important to point out are:

1. Twitter’s Terms of Service at that time basically only outlawed “unsolicited replies,” nothing that really attacked targeted spam.
2. Twitter’s anti-spam stance did exist in writing (only in the help site,) but I do not think they were actively enforcing their policies.
3. My recommendations were contextual and, unless you looked at my bot’s timeline and tweet count, looked legitimate (most of the time.) In other words, I was tweeting book suggestions to people who were already talking about books.
4. I recorded the usernames of everyone I sent recommendations to and would only @mention them once.
5. I built in a “chattiness” rate limiting function. This was to distribute my spam throughout a whole hour (due to Twitter’s rate limiting) more than anything.

While it only lasted a short while, I had alot of fun and made a little bit of money spamming Twitter.

The second re-incarnation of this project turned into BookSuggest, a website for recommending books based on a person’s Twitter feed. I haven’t put alot of effort into promoting it, but my conversion rate is much lower now that I’m not pushing the links in anyone’s face.

Try it out and comment here – what did BookSuggest tell YOU to read?

This entry is part 3 of 8 in the seriesIntro to Financial Reporting

Previously, we discussed the various regulations and regulatory bodies that govern financial reporting. We will now turn to the Generally Accepted Accounting Principles (GAAP) to explain the basic principles used in accounting. In particular, we will discuss the cost, revenue recognition, matching, and full disclosure principles.

While it may sound redundant, the cost principle means that “accounting information is based on an actual cost” (Wild, Shaw, & Chiappetta, 2009, p. 9). In other words, everything is treated as having the value of what was paid for it. So what happens when businesses make a trade or don’t purchase with cash (businesses have been known to buy each other with a mix of cash, stocks, and bonds)? “If something besides cash is exchanged … cost is measured as the cash value of what is given up or received” (Wild, Shaw, & Chiappetta, 2009, p. 10). That caveat here is that if you buy something and get a good deal, such as buying a $7000 asset for $5000, the item will be recognized in your accounting system as having a value of $5000. This is to ensure that the accounting information remains objective (Wild, Shaw, & Chiappetta, 2009, p. 10). Next, we will look at the revenue recognition principle.

The revenue recognition principle determines how and when a company will recognize (record) revenue (Wild, Shaw, & Chiappetta, 2009, p. 10). The primary concept of the revenue recognition principle is that “Revenue is recognized when earned” (Wild, Shaw, & Chiappetta, 2009, p. 10). This doesn’t necessarily mean when the customer or client pays for their good or service, but when the good or service is actually sold (such as on credit). For example, if I configured someone’s network (a service) at an hourly rate, I would be required to recognize and record this earned revenue as soon as the work is done; this is usually done crediting accounts receivable (most accounting software does this automatically when generating an invoice.) This principle is intended to keep companies from recognizing revenue too early to look more profitable while also ensuring that they don’t recognize revenue too late to look less profitable than they really are (Wild, Shaw, & Chiappetta, 2009, p. 10). Now let’s look to the matching principle.

The matching principle dictates that a company must report its expenses in the period that they generated the revenue reported (Wild, Shaw, & Chiappetta, 2009, p. 10). Let’s say, for example, that a company buys 10 pounds of raw material, uses it to make 10 widgets, and then sells 5 of those widgets. Under the matching principle, the company would report the expenses (or, in this case, Cost of Goods Sold) incurred to make the 5 widgets it sold. The remaining 5 (that are now sitting in inventory), would not have their expenses/COGS reported until they too were sold. Finally, we turn to the full disclosure principle.

The full disclosure principle is probably the most basic yet most important principle. The full disclosure principle states that a company must “report the details behind financial statements that would impact users’ decisions” (Wild, Shaw, & Chiappetta, 2009, p. 10). Oftentimes, these details are reported in footnotes of a company’s financial statements or annual reports (Wild, Shaw, & Chiappetta, 2009, p. 10). An example of this from current events is Dell’s recent trouble with the SEC. Dell’s recent trouble was partially the result of receiving money from CPU manufacturer Intel and disguising that money as sales (why they hid it is another topic entirely). Users of Dell’s financial reports were led to believe that this extra money was the result of sales. When Intel stopped paying Dell this “incentive money,” Dell then took extra steps to falsify their financial statements to hide the fact that their revenue decreased. Save for the anti-trust violation with Intel, if Dell had just fully disclosed the revenue it was receiving from Intel they may have never felt the pressure to hide the fact that the payments stopped.

In conclusion, the Generally Accepted Accounting Principles (GAAP) are made up of four basic principles. In particular, we discussed the cost, revenue recognition, matching, and full disclosure principles.

Chiappetta, B., Shaw, K., Wild, J. (2009). Principles of Financial Accounting (19th ed.). McGraw-Hill/Irwin.

This entry is part 6 of 8 in the series Intro to Financial Reporting

Enron was an energy trading company. Rather than generate energy themselves, Enron made its money by buying and selling energy contracts. At one point, Enron was considered one of the most innovative companies on the market and reported sales of over $100 billion one year. Soon after, the company then reported losses of $618 million loss in the third quarter of 2001 (Nielsen, 2002, para. 1). A little later, the firm filed for Chapter 11 bankruptcy. “The company’s Chapter 11 filing [left] banks, pension plans and other lenders with at least $5 billion at risk. More than 4,000 Enron employees lost their jobs and 401(k) savings. The collapse reverberated the stock market, which dropped some $200 billion in value since Enron’s Dec. 2 filing, amid fears that other Enrons are lurking out there” (Jaffe, 2002, para. 3).

So how does a company go from reporting over $100 billion dollars in sales when they are really on the verge of bankruptcy? The answer lies in the deceptive accounting practices and outright fraud that Enron’s executives and auditors took part in. One factor is the massive amount of debt that Enron was hiding from its balance sheet. How does one go about hiding debt? “At the heart of Enron’s demise was the creation of partnerships with shell companies, many with names like Chewco and JEDI, inspired by Star Wars characters. These shell companies, run by Enron executives who profited richly from them, allowed Enron to keep hundreds of millions of dollars in debt off its books” (Nielsen, 2002, para. 4). At the same time, Enron’s board members, regulators, analysts, auditors, and even politicians looked the other way while Enron committed its fraud (Kadlec, 2001, para. 4). Enron’s auditing firm even went as far as to order their employees to shred all of the documents they used to do Enron’s audits! In the wake of Enron’s collapse, the general public demanded that the politicians step up, create, and enforce corporate law. This allowed for the creation of the Sarbanes-Oxley Act of 2002 which is still in effect today!

Jaffe, Stephen. TIME. (2002). How Fastow Helped Enron fall. Retrieved from 

Kadlec, Daniel. TIME. (2001). Power Failure. Retrieved from 

Nielsen, James. TIME. (2002). Enron: Who’s Accountable? Retrieved from 

This entry is part 7 of 8 in the series Intro to Financial Reporting

After Enron, Global Crossing, and Xerox, we witnessed the creation of new regulation (the Sarbanes-Oxley Act of 2002), the establishment of a new oversight/regulatory organization, and increased pressure from the government on big players in the market. As we find ourselves in the midst of the Financial Crisis of 2007 (and onward,) we are seeing this same pattern – new legislation, a new regulatory organization, and increased pressure from the government on big players in the market.

In news today, we see the Restoring American Financial Stability Act of 2010 and the Wall Street Reform and Consumer Protection Act of 2009 on the desk of the President. This 2,300-page bill will “ban high-risk trading inside the banks and put in end to conflicts of interest” and “ban steering payments, liar loans, and prepayment penalties and give Americans … transparency” (Nichols, 2010, para. 4-5). But that isn’t all. These new acts would also create yet another oversight/regulatory organization called the Consumer Financial Protection Bureau to regulate mortgages, credit cards, and other financial consumer products (Hall, 2010, fig. 1). Not only does this act establish a new bureau, it also creates a “council of regulators” led by the Treasury to monitor “threats to the financial situation” (Hall, 2010, fig. 1).

And as if you couldn’t already guess, today in the news we see increased pressure from the government on big players in the market including Goldman Sachs, Dell, and AIG. Goldman Sachs agreed to settle by paying a $550 million dollar fine – the largest fine ever charged by the SEC after being accused of securities fraud (Craig, Scannel, 2010, para. 3). Dell admitted to accounting fraud back in 2007 and agreed on a settlement with the SEC today – The company must pay a $100 fine and restate its earnings for the 2003 to 2006 accounting periods (Ogg, 2010, para. 2-3). What Dell was caught doing was violating GAAP, the Global Accepted Accounting Principles, by “fudging” the timing of recognized expenses and income to meet analyst’s quarterly earnings forecasts (Ogg, 2010, para. 1,4). And if that isn’t enough, the AIG settled with the Ohio State Attorney General with a $725 million fine (AFP, 2010, para. 1). The AIG was also accused of accounting fraud to achieve stock price manipulation (AFP, 2010, para. 2).

As we can clearly see, after each financial crisis the response is likely to be the same.

AFP. (2010). AIG to Pay 725 Million Dollars to Settle US Fraud Lawsuit. Retried from http://www.google.com/hostednews/afp/article/ALeqM5h0MtX-uuAB1z2f3dcmONOLRCIWaw

Craig, Susanne, Kara Scannel. The Wall Street Journal. (2010). SEC Split Over Goldman Deal. Retrieved from http://online.wsj.com/article/SB10001424052748704229004575371601322076426.html?mod=WSJ_hpp_LEFTTopStories

Hall, Kevin. McClatchy Newpapers. (2010). What’s this big finance-regulation overhaul really do? Retrived from http://www.mcclatchydc.com/2010/07/15/97609/whats-this-big-finance-regulation.html

Ogg, Erica. CNET. (2010). Dell to Restate Earnings Due to Accounting Fraud. Retrieved from http://news.cnet.com/8301-31021_3-20007390-260.html

Ogg, Erica. CNET. (2010). Goals Led Dell to Cook the Books. Retrieved from http://news.cnet.com/Goals-led-Dell-to-cook-the-books/2100-1014_3-6203071.html?tag=mncol;txt

Nichols, John. The Nation. (2010). Financial Reform Passes, But What Does That Mean? Retrieved from http://www.thenation.com/blog/37541/financial-reform-passes-what-does-mean

This entry is part 5 of 8 in the seriesIntro to Financial Reporting

The Sarbanes-Oxley Act of 2002, also known as the SOX Act, was created in response to the series of misleading and outright fraudulent activity of big business in the 1990s (Lasher, 2008, p. 187). Essentially, multiple publicly-traded businesses jacked up their stock prices by “publishing false or deceptive financial statements” (Lasher, 2008, p. 187). The most notable company to crash was Enron, followed by Global Crossing (parent of MCI,) and Xerox; later, almost one thousand publicly traded companies restated their financial statements (Lasher, 2008, p. 187). This resulted in almost $6 trillion of stock market value disappearing (Lasher, 2008, p. 187)! In response to these events, Congress drafted and passed the Sarbanes-Oxley Act (SOX) of 2002.

Next, we look to the provisions of the Sarbanes-Oxley Act of 2002 to see how they impact the way firms prepare their financial statements. The first and most major way the SOX Act impacted financial reporting was that it ended self-regulation of the public accounting industry (Lasher, 2008, p. 190). The SOX Act achieved this by establishing the Public Company Accounting Oversight Board (PCAOB,) an independent, non-profit organization (Lasher, 2008, p. 190). The PCAOB is given authority and empowered by the Securities Exchange Commission (SEC) to regulate and enforce these regulations of the accounting industry (Lasher, 2008, p. 190). Under the regulations of the SOX Act and the PCAOB, it is now required for all accounting firms to be registered and illegal for an unregistered firm to issue audit reports for publicly-traded companies (Lasher, 2008, p. 190). The PCAOB is empowered and directed to perform investigations of questionable accounting practices, hold disciplinary hearings, and impose sanctions upon firms and individuals who auditors who are caught letting wrongdoings “slide” (Lasher, 2008, p. 190). Another way the SOX Act effectively restores the integrity of financial statements is by removing a very large conflict of interest that existed in the 1990s. This conflict of interest existed for accounting firms that also provided consulting services to their clients; under the SOX Act, these types of relationships became illegal and firms that had audit clients could no longer offer these same clients additional consulting services (Lasher, 2008, p. 190). Another major conflict of interest that was removed was the relationships between auditors and the audited firm’s CEO and CFO. Instead of reporting to the firm’s executives, auditors now reported to an “audit committee” of the firm’s Board of Directors; At least one member of which is legally required to be a financial expert (Lasher, 2008, p. 190). Finally, partners of an auditing firm can only supervise the same client for five years at a time (Lasher, 2008, p. 191). The final main problem that the SOX Act addresses is the potential conflict of interest between a firm and its executives, also known as the agency problem (Lasher, 2008, p. 192). The SOX Act achieved this by requiring CEOs and CFOs to certify that they reviewed their firm’s financial statements, that they are true to the best of their knowledge, to also certify that they are personally responsible for their firm’s internal financial controls, and by requiring that company executives repay bonuses and capital gains from stock sales if they follow within 12 months of the issue of financial statements and the gains are made as a “result of misconduct” (Lasher, 2008, p. 191).

Lasher, William R. (2008). Practical Financial Management (5th ed.). Thomson South-Western.

This entry is part 4 of 8 in the series Intro to Financial Reporting

As previously discussed, financial statements can sometimes have their own “dialect,” in a manner of speaking. Additionally, financial statements can be subject to accounting fraud because management is usually rewarded based on their firm’s performance which is measured based on the financial statements that they prepare. It is for these reasons that oversight and regulation of financial reporting is necessary. In addition to the Generally Accepted Accounting Principles (GAAP,) financial reporting is regulated by organizations such as the Securities Exchange Commission (SEC), the Financial Accounting Standards Board (FASB), and the International Accounting Standards Board (IASB) (Wild, Shaw, & Chiappetta, 2009, p. 9).

The mission of the Securities Exchange Commission, or SEC (2010), is to “protect investors, maintain fair, orderly, and efficient markets, and facilitate capital formation” (para. 1). The SEC maintains that all investors, whether larger or small, should have all of the basic information of a security required to make investment decisions (para 6). As a result, the SEC requires all publicly traded companies to disclose “meaningful financial and other information to the public” (para 6). The SEC was formed under the Securities Act of 1933 and the Securities Exchange Act of 1934 in the wake of the great depression to restore confidence in the stock markets (para 16).

Next, we have the Financial Accounting Standards Board, or FASB. Similar to the SEC, the FASB aims ensure that investors have useful information for making investment decisions. More specifically, the FASB’s mission is to “establish and improve standards of financial accounting and reporting that foster financial reporting by nongovernmental entities that provides decision-useful information to investors and other users of financial reports” (para 3). The FASB notes on their website that while the SEC has the legal authority to establish the financial accounting and reporting standards, the SEC’s position is to rely on private organizations, like the FASB, for this function (para 2).

Finally, we have the International Accounting Standards Board, or IASB. The IASB (2010) is responsible for developing and publishing International Financial Reporting Standards (or IFRSes,) ensuring that these standards are enforceable, and promoting their use (para 1).

To conclude, because financial statements can sometimes have their own “dialect” and are subject to accounting fraud there are various organizations that are required to regulate the preparing and publishing of them. The major organizations responsible for this are the Securities Exchange Commission (SEC), the Financial Accounting Standards Board (FASB), and the International Accounting Standards Board (IASB).

Chiappetta, B., Shaw, K., Wild, J. (2009). Principles of Financial Accounting (19th ed.). McGraw-Hill/Irwin.
Financial Accounting Standards Board [FASB]. Facts about FASB. Retrieved from http://www.fasb.org/jsp/FASB/Page/SectionPage&cid=1176154526495
International Accounting Standards Board [IASB]. (2010). About the IASCF Foundation and the IASB. Retrieved from http://www.iasb.org/The organisation/IASCF and IASB.htm
Securities Exchange Commission [SEC]. (2010). How the SEC Protects Investors, Maintains Market Integrity, and Facilitates Capital Formation. Retrieved from http://www.sec.gov/about/whatwedo.shtml

This entry is part 1 of 7 in the series Intro to Financial Reporting

If “accounting is the language of finance” (Lasher, 2008, p. 9) then financial reporting is the “communication of financial information useful for making investment, credit, and other business decisions” (Wild, Shaw, & Chiappetta, 2009, p. 681) Such communications include general purpose financial statements such as income statements, balance sheets, equity reports, cash flow reports, and notes to these statements. Additionally, items such as SEC filings, press releases, meeting minutes, and auditor’s reports are also included in financial reporting (Wild, Shaw, & Chiappetta, 2009, p. 681). Many financial reports, or the accounts and data they represent, are subject to various regulations and standards from organizations such as the Securities Exchange Commission (SEC), the Financial Accounting Standards Board (FASB), and the International Accounting Standards Board (IASB) (Wild, Shaw, & Chiappetta, 2009, p. 9). Much like any language, financial statements could have their own “dialect” so to speak. For example, knowing about the use of cash-based accounting versus accrual based accounting could impact some very serious business or investment decisions. The various regulations, standards, and Generally Accepted Accounting Principles (GAAP) helps to make sure we’re all on the same page.

In the broad sense of the term, everyone uses financial reports! We all receive receipts when we make purchases from stores and we all receive bills. In a sense, these are both financial reports that communicate to us the status of our accounts or individual transactions. When we focus on business, however, we can more easily focus on managers, investors, creditors, and even the government. Managers use financial reports to make business decisions. For example, if a manager of a manufacturing firm saw from internal financial and inventory reports that product returns were high then that manager might push for increased quality control. Investors and potential investors alike use general-purpose financial reports so frequently that companies often release them together in a bundle called “investor reports,” “annual reports,” or “shareholder reports.” Investors would use this information to help make a decision about whether they will buy, sell, or hold onto a particular company’s stock. Another large group of people who use financial reports are creditors. A creditor would use financial reports to determine their risk in loaning money to a particular company.

Lasher, William R. (2008). Practical Financial Management (5th ed.). Thomson South-Western.
Chiappetta, B., Shaw, K., Wild, J. (2009). Principles of Financial Accounting (19th ed.). McGraw-Hill/Irwin.

This entry is part 2 of 8 in the series Intro to Financial Reporting

According to Lasher’s “Practical Financial Management,” the three financial statements of interest to us are the income statement, the balance sheet, and the statement of cash flows (Lasher, 2008, p. 26).

The income statement’s name is pretty self-explanatory. That is, the income statement reports “how much money a company has earned [and spent] during the accounting period” (Lasher, 208, p. 28). A public, or external, income statement may be very vague while a income statement that is used internally may include much more detail. Despite these differences, the net income will be reported as the same. Net income goes by some other names as well, including “earnings” or “profit” (Lasher, 2008, p. 30). When a company loses money, net income is referred to as net loss. One important thing to note is that “net income is not equivalent to cash in the firm’s pocket” (Lasher, 2008, p. 30). This is because many businesses make sales on credit.

While the income statement shows the movement of money within a firm, a balance sheet depicts a snapshot of “everything a company owns and everything it owes at a moment in time” (Lasher, 2008, p. 30). This includes a firm’s assets, liabilities, and equity. For those who aren’t familiar, this depicts the entire accounting equation: assets = liabilities equity (Lasher, 2008, p. 30). Assets are “things,” whether physical of financial, that a company owns. This includes everything from real estate to manufacturing equipment and even stocks or bonds in other companies. Assets are listed in descending order of liquidity on the balance sheet (Lasher, 2008, p. 31). Meanwhile, liabilities includes money that the company owes, ie: debt. Liabilities are listed in order of when they are due. The final portion of the equation is equity. Equity is money put into the business by its owners. . The two types of equity are direct investment and retained earnings (Lasher, 2008, p. 39).

The statement of cash flows helps to figure out how much cash the company is really making in the short run (Lasher, 2008, p. 30). In other words, the statement of cash flows shows “where the firm’s money came from and what it was spent on” (Lasher, 2008, p. 70). Another name for the statement of cash flows is the “statement of changes in financial position.”

Lasher, William R. (2008). Practical Financial Management (5th ed.). Thomson South-Western.

This is the series Intro to Financial Reporting

At the date of publication, I was taking summer classes for Sociology and Principles of Finance. As part of the class, I was required to write a one page paper each week on a topic I chose. I chose Financial Reporting. At the end of the semester, these papers were concatenated and turned into one monstrous paper. japherwocky suggested post the papers to a blog, so here they are.

1. What Are Some of the Main Financial Documents Used in Financial Reporting?
2. What Is Financial Reporting and Who Uses Financial Reports?
3. What Are the Generally Accepted Accounting Principles (GAAP)?
4. Who Regulates Financial Reporting?
5. Why Was the Sarbanes Oxley Act of 2002 Created and How Does it Impact Financial Reporting Today?
6. Who Was Enron and Why Is This Important to Financial Reporting Today?
7. Financial Crises: And the Cycle Continues

Today, I’m going to show you how to start up your Tornado apps at boot using upstart. For path names, I’m assuming some modern version of Ubuntu, such as 9.10 or higher. I’m also assuming that you have some project with an executable Python script that fires up the Tornado app. I usually have one file in all of my projects called web.py.

The important part of this file looks like this (mind the application and port variables):

1
2
3
4

if __name__ == "__main__":
	http_server = tornado.httpserver.HTTPServer(application)
	http_server.listen(port)
	tornado.ioloop.IOLoop.instance().start()

Got that in place? Good. Next, make a system user for your project to run as. In my example, I’ll be using projectuser as the username. Creating this user can be done like so:

1

sudo useradd --system --user-group projectuser

Finally, create the file /etc/init/web.conf. It’s very important that this filename ends in .conf. The contents of this file should be as follows:

1
2
3
4
5
6
7
8
9

\# torando project
start on runlevel 2
stop on runlevel \[!2\]
respawn

setuid projectuser
setgid projectuser

exec /path/to/project/web.py

That’s it! You can start your Tornado app by entering the command: sudo start web. If successful, you should see output similar to: ` ** web start/running, process 28058. You can also stop your projects using the command sudo stop web`.