texts: Modular widget that minimizes js and css scope pollution

texts:
So, my goal with this is to create a simple login widget that does as close to nothing to the global scope as possible. The login mechanism is less important than the encapsulation. The whole script is wrapped in an immediately invoked function expression, no ids are used on elements, document.currentScript is used for identification of the widget, nothing is written to window, and a custom event on the form element is used to communicate with outside scripts.

<form class="login popout">
  Login
  <input type="username" name="username" placeholder="Username"/>
  <input type="password" name="password" placeholder="Password"/>
  <output class="error"></output>
  <button type="submit">Send</button>

</form>

I wrote this to be compatible with all current-gen browsers as of 2016-05-26. Document.currentScript specifically should cause the most problems for older browsers. Is there any way to tighten this up further? Is this excessive?
My motivation was getting a totally reusable and self-reliant widget that doesn't rely on shadow DOM or other black magic, and doesn't care what the rest of the page is doing.


texts: Better way to calculate double-scattering diffraction using cartesian product of arrays?

texts:
This script calculates points in reciprocal space for hexagonal 2D lattices, then uses the cartesian product from itertools to add each vector from one lattice to all of the vectors of the other in the line
np.array([a+b for a, b in list(itertools.product(p1.T, p2.T))])

It's slow right now because as written its instantiating millions of tiny numpy arrays.
I'm aware of:

Cartesian product of x and y array points into single array of 2D points
Using numpy to build an array of all combinations of two arrays
cartesian products in numPy

and I suspect there's some way to do this, possibly using np.meshgrid  or np.mgrid that's faster, uses less memory and looks cleaner, but I can not figure out how.
I will use the output in an optimization loop matching these positions to measured positions, so it needs to be callable several hundred times in a row, so reusing large array spaces rather than instantiating and garbage collecting them might have some advantages.
 click for larger
import numpy as np
import matplotlib.pyplot as plt
import itertools

def rotatem(xy, rot):
    r3o2, twopi, to_degs, to_rads = np.sqrt(3)/2., 2*np.pi, 180/np.pi, np.pi/180
    c, s = [f(to_rads*rot) for f in (np.cos, np.sin)]
    x, y = xy
    xr = c*x - s*y
    yr = c*y + s*x
    return np.vstack((xr, yr))

def get_points(a=1.0, nmax=5, rot=0):
    r3o2, twopi, to_degs, to_rads = np.sqrt(3)/2., 2*np.pi, 180/np.pi, np.pi/180
    g = twopi / (r3o2 * a)
    i = np.arange(-nmax, nmax+1)
    I, J = [thing.flatten() for thing in np.meshgrid(i, i)]
    keep = np.abs(I + J) <= nmax
    I, J = [thing[keep] for thing in (I, J)]
    xy = np.vstack((I+0.5*J, r3o2*J))
    return g * rotatem(xy, rot=rot)

r3o2, twopi, to_degs, to_rads = np.sqrt(3)/2., 2*np.pi, 180/np.pi, np.pi/180

a1, a2, rot = 1.0, 2**0.2, 22

p1 = get_points(a=a1, nmax=20)
p2 = get_points(a=a2, nmax=20, rot=rot)
p3 = get_points(a=a2, nmax=20, rot=-rot)

d12 = np.array([a+b for a, b in list(itertools.product(p1.T, p2.T))])
d13 = np.array([a+b for a, b in list(itertools.product(p1.T, p3.T))])
d12, d13 = [d[((d**2).sum(axis=1)<4.)] for d in (d12, d13)]

if True:
    plt.figure()
    for d in (d12, d13):
        plt.plot(*d.T, 'o', ms=2)
    plt.gca().set_aspect('equal')
    plt.show()


texts:
public static int lastDigit(int a) {
    a = a % 10;
    if (a <= 0){
    a *= -1;
    }
    return a;
}

This is my solution. The course hasn't taught if else yet, so I'm wondering if there is a way to do it without the if else.

public static int lastDigit(int a) {
a = a % 10;
return a;
}

The problem with this is that it will not print the last digit of a negative number as a positive.


texts:
I am a 3rd-year computer science undergraduate. One of my university lecturers has developed his own page for students to submit work. It came up that one student was accused of hacking (sic) by the system. The problem was the characters used in the comment field.

The lecturer claimed that he needed to filter out certain characters in order to stop attacks, and that it was a known problem with PHP. (His system had been broken by students before). I am suspecting poor programming, as he claims that the strings are being executed by PHP.

He told me to try writing a PHP program with a form text field, then input PHP code into the text form. He expects that there to be a vulnerability without filtering the input for bad chars.

<?php

if(isset($_POST['data']) ){
    if( file_exists("../post.sqlite") ){
        $db = new SQLite3("../post.sqlite");

        if( isset($_POST['password']) && $_POST['password'] === "correct horse battery staple" ){
            $statement = $db->prepare("INSERT INTO posts( id, post ) VALUES ( NULL, ? )");
            $statement->bindValue(1, $_POST['data'], SQLITE3_TEXT);
            $statement->execute();
        }

    }
    else {
        $db = new SQLite3("../post.sqlite");

        $db->exec("CREATE TABLE posts( id INTEGER PRIMARY KEY, post TEXT )");
    }
}

function posts(){
    if( file_exists("../post.sqlite") ){
        $db = new SQLite3("../post.sqlite");

        $query = $db->query("SELECT post from posts ORDER BY id DESC");

        while( $row = $query->fetchArray(SQLITE3_ASSOC)){
            print( "
" );
            print( htmlentities( $row['post'] ) );
            print( "

" );
        }
    }
}

?>
<!DOCTYPE html>

    <title>Try This</title>

        <form action="post.php" method="post">
            <textarea rows="10" cols="80" name="data"></textarea>

            <input type="submit" value="Post" />

            Secret key: <input type="text" name="password" />
        </form>

            <?php posts(); ?>

Without the htmlentities function, I know it would have client side vulnerabilities (running malicious scripts). SQL injection is covered (prepared statements). I don't see how the program can be exploited server side. What (if anything) am I missing?


texts: Google Custom Search placeholder function in JavaScript

texts:
Problem:

We have a Google Custom Search function that replaces the input placeholder attribute and it works okay.

Would you be so kind and help me with reviewing it, and if possible, provide an efficient function to replace this function with (also hoping that it would work with older version of IE and other browsers, not very necessary though):

Placeholder Script:

// google search placeholder
(function() {
    var cx = '!!!!!!!!!!!!!!!!!!!';
    var gcse = document.createElement('script');
    gcse.type = 'text/javascript';
    gcse.async = true;
    gcse.src = 'https://cse.google.com/cse.js?cx='+ cx;
    var s = document.getElementsByTagName('script')[0];
    s.parentNode.insertBefore(gcse, s);
})();

window.onload = function(){
    document.getElementById('gsc-i-id1').placeholder = 'e.g., United Stocks AMD';
};

where cx is a dynamic variable string parameter, defined as the custom search engine ID to use for a request (Custom Search API, ID for custom search).

Result Display Script:

<gcse:searchresults-only></gcse:searchresults-only>

Popular Query Script:

Added CSS:

/*google custom search*/
.cse .gsc-search-button input.gsc-search-button-v2,input.gsc-search-button-v2{height:26px!important;margin-top:0!important;min-width:13px!important;padding:5px 26px!important;width:68px!important}
.cse .gsc-search-button-v2,.gsc-search-button-v2{box-sizing:content-box;min-width:13px!important;min-height:16px!important;cursor:pointer;border-radius:20px}
.gsc-search-button-v2 svg{vertical-align:middle}
.gs-title{line-height:normal!important}
.gsc-search-box-tools .gsc-search-box .gsc-input{padding:5px!important;color:#4169E1!important;border-radius:20px}
.gsc-input-box{background:none!important;border:none!important}

Screenshot Before Search:

Screenshot After Search:

Screenshot for "Code Review" search:


texts:
I have written this code after studying from Introduction to Algorithm and from GeeksForGeeks. I know there is a lot to improve because I don't know much C++11. Please help me to improve this code.

#include <iostream>
#include <algorithm>
#include <vector>
#include <queue>
#include <string>

class HuffmanCodes
{
 struct Node
 {
  char data;
  size_t freq;
  Node* left;
  Node* right;
  Node(char data, size_t freq) : data(data),
                                 freq(freq),
                                 left(nullptr),
                                 right(nullptr)
                                 {}
 ~Node()
 {
   delete left;
   delete right;
 }
 };

 struct compare
 {
  bool operator()(Node* l, Node* r)
  {
    return (l->freq > r->freq);
  }
};

Node* top;

void printCode(Node* root, std::string str)
{
  if(root == nullptr)
   return;

 if(root->data == '$')
 {
  printCode(root->left, str + "0");
  printCode(root->right, str + "1");
 }

 if(root->data != '$')
 {
   std::cout << root->data << " : " << str << "\n";
   printCode(root->left, str + "0");
   printCode(root->right, str + "1");
 }
}

public:
  HuffmanCodes() {};
  ~HuffmanCodes()
  {
    delete top;
  }
  void GenerateCode(std::vector<char>& data, std::vector<size_t>& freq, size_t size)
  {
   Node* left;
   Node* right;
   std::priority_queue<Node*, std::vector<Node*>, compare > minHeap;

   for(size_t i = 0; i < size; ++i)
   {
      minHeap.push(new Node(data[i], freq[i]));
   }

    while(minHeap.size() != 1)
    {
      left = minHeap.top();
      minHeap.pop();

      right = minHeap.top();
      minHeap.pop();

      top = new Node('$', left->freq + right->freq);
      top->left  = left;
      top->right = right;
      minHeap.push(top);
     }
    printCode(minHeap.top(), "");
 }
};

 int main()
 {
  HuffmanCodes set1;
  std::vector<char> data({'d', 'e', 'b', 'c', 'a', 'f'});
  std::vector<size_t> freq({16, 9, 13, 12, 45, 5});
  size_t size = data.size();
  set1.GenerateCode(data, freq, size);

  return 0;
}


texts:
I'm teaching myself Python and wanted to share this problem as my current solution feels inelegant but I'm not sure of the best way to refactor. If this sort of question isn't appropriate, leave a comment and I'll remove it.

I'm writing a generic library to handle web API requests. The aim is to have a generic core that can be fed with URL templates and parameters - so each new API is just a wrapper rather than duplicating core logic like paging, parsing, etc.

Trouble is, each URL has a completely different structure:

Facebook feeds have an ID variable in the URL endpoint - /{id}/feed/
Facebook insights require per-request parameters as well eg /{id}/insights/page_fans?since={timestamp}&until={timestamp}
Instagram user search requires the ID to be a query parameter - /users/search?q={}

Then all URLs need additional query parameters like access_token which is static.

I had started with a simple string.format() to replace the variables, but this doesn't work with dynamic parameters.

New model is to build the URL from a mixture of endpoint, static parameters and dynamic parameters. I'm then using itertools.product to work out the permutations and then encode the URLs.

Current thinking is to create a set of methods for each common problem (date ranges, etc) and just call each one as required during the setup wrapper.

Here's an example of a working wrapper - can you see any more Pythonic ways to handle this situation?

def insights_impressions_unique(ids, access_token, date_start=None,
                                date_end=None, period="day"):
    if period not in ["day", "week", "days_28"]:
        raise ValueError("Period must be 'day','week' or 'days_28'")

    # Build basic URLs
    _api_domain = "https://graph.facebook.com"
    _api_endpoint = "/v2.2/{}/insights/page_impressions_unique"
    url_template = "{}{}".format(_api_domain, _api_endpoint)

    _formatted_urls = [url_template.format(_id) for _id in ids]

    # Move on to paramaters
    _base_params = [{"access_token": access_token,
                    "period": period}, ]

    # Build date periods
    _utcnow = datetime.datetime.utcnow().replace(tzinfo=pytz.utc)
    if date_end is None:
        date_end = _utcnow
    if date_start is None:
        date_start = _utcnow - datetime.datetime.timedelta(days=-7)
    _date_windows = utils.api_call_time_windows(date_start, date_end, freq=7)
    _date_params = [{'since': x[0], 'until': x[1]} for x in _date_windows]

    _combined_params = product(_base_params, _date_params)

    list_of_params = list()
    for i in _combined_params:
        params = {}
        for d in i:
            for k, v in d.iteritems():
                params[k] = v
        list_of_params.append(urllib.urlencode(params))

    _components = product(_formatted_urls, list_of_params)
    _finalised_urls = ["{0}?{1}".format(_u, _p) for _u, _p in _components]

    manager = RequestManager(urls=_finalised_urls, pagination=False)
    return manager

Full code on this dev branch on GitHub.


texts: Regex version of strip() - Ch. 7 Automate the Boring Stuff

texts:
Here is a practice exercise — Regex version of strip() \$-\$

  Write a function that takes a string and does the same thing as the
  strip() string method. If no other arguments are passed other than the
  string to strip, then whitespace characters will be removed from the
  beginning and end of the string. Otherwise, the characters specified
  in the second argument to the function will be removed from the
  string.

I have written the following code. Is there any better way to write it? Any feedback is highly appreciated.

import re

def regex_strip(s, chars = None):

    if chars == None:
        strip_left = re.compile(r'^\s*')
        strip_right = re.compile(r'\s*$')

        s = re.sub(strip_left, "", s)
        s = re.sub(strip_right, "", s)
    else:
        strip_left = re.compile(r'^[' + re.escape(chars) + r']*')
        strip_right = re.compile(r'[' + re.escape(chars) + r']*$')
        s = re.sub(strip_left, "", s)
        s = re.sub(strip_right, "", s)
    return s

Here is an example output -

s = '.*    alphabetatheta   *4453   +-'
print(regex_strip(s, '.+-*'))

>>>    alphabetatheta      *4453


texts:
I've implemented a promises in mongoose to reduce the amount of callback hell but I am not sure whether my technique is correct.

The scenario:

Before I can save my schema I want to run some functions in specific order. These functions update other collections in mongodb. So I have used promises to implement this.

Code: (using promises)

// before saving do this
return incrementSessionTypesTotal(req, res)
.then(
    // before saving do this
    incrementSessionsTypes(req, res, newSurvey))
   //  now save my schema!
.then(newSurvey.save(function(err, data) {
    if (err) {
        sendJsonResponse(res, 400, err)
    } else {
        sendJsonResponse(res, 200, data);
    }
}));

Code:

function incrementSessionTypesTotal(req, res) {

return Total.findOne({dP: "xyz"}, function(err, doc) {
    if (err) {
        sendJsonResponse(res, 401, err)
    } else if (doc) {

            if (req.body.q1 === "poor") {
                doc.poorSessions += 1;
            } else if (req.body.q1 === "ok") {
                doc.okSessions += 1;
            } else if (req.body.q1 === "well") {
                doc.wellSessions += 1;
            } else if (req.body.q1 === "very well") {
                doc.veryWellSessions += 1;
            }
         }

     doc.save(function(err, data) {
        if (err) {
            sendJsonResponse(res, 400, err)
        } else {
            return data;
        }
    });

    }).exec();
 }

Is this implementation okay? I have not really used promises with mongoose before so I am not sure whether this is correct. I have tested the code and it works but I would like some feedback.


texts: Web-scraping investment fund data into Pandas

texts:
Just starting out trying to do some python for data analysis. I am a total beginner and have figure out how to brute force somethings, but I know it is inefficient but don't know how to do it without messing up what i have.

I have multiple webpages to scrape and then store data in a data frame. The code is identical for all of the pages. How do I set it up as a routine instead just repeating the same code over and over again.

As an example the two urls are: https://etfdb.com/etf/IWD/ https://etfdb.com/etf/IWF/

The html is identical so the web scraping working exactly the same for both.

Once scraped, I want them in a single data frame.

The below works, but is me taking the least sophisticated approach since I know so little. The actual code is likely not pretty but it works.

Any help appreciated on how this should be improved.

import bs4
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
import pandas as pd
import numpy as np
from IPython.display import display

iwd_url = 'https://etfdb.com/etf/IWD/'
uClient = uReq(iwd_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")

#Isolate header to get name and symbol
h1 = page_soup.h1

#Isolate stock symbol
title = h1.findAll("span",{"class":"label-primary"})
titlet = title[0].text

#print(titlet)

#strip space and line break
strip1 = h1.text.strip()

#strip stock symbol
strip2 = strip1.strip(titlet)

#strip remaining line break
strip3 = strip2.strip()

#print(strip3)

IWD = page_soup.findAll("table",{"class":"chart base-table"})[1]
#Create lists to fill
sectordata=[]
sectorname=[]
sectorweight=[]
for row in IWD.findAll("td"):

    sectordata.append(row.text)
#list created

#Assign every other value to proper list to get 2 columns
sectorname = sectordata[::2]
sectorweight = sectordata[1::2]

#Insert name/symbol for clarification/validation
sectorweight.insert(0,titlet)
sectorname.insert(0,strip3)

# create empty data frame in pandas
df = pd.DataFrame()
#Add the first column to the empty dataframe.
df['Sector']  = sectorname

#Now add the second column.
df['Weight'] = sectorweight
##display(df)

### NEXT

iwf_url = 'https://etfdb.com/etf/IWF/'
uClient = uReq(iwf_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")

#Isolate header to get name and symbol
h1 = page_soup.h1

#Isolate stock symbol
title = h1.findAll("span",{"class":"label-primary"})
titlet = title[0].text

#print(titlet)

#strip space and line break
strip1 = h1.text.strip()

#strip stock symbol
strip2 = strip1.strip(titlet)

#strip remaining line break
strip3 = strip2.strip()

#print(strip3)

IWD = page_soup.findAll("table",{"class":"chart base-table"})[1]
#Create lists to fill
sectordata=[]
sectorname=[]
sectorweight=[]
for row in IWD.findAll("td"):

    sectordata.append(row.text)
#list created

#Assign every other value to proper list to get 2 columns
sectorname = sectordata[::2]
sectorweight = sectordata[1::2]

#Insert name/symbol for clarification/validation
sectorweight.insert(0,titlet)
sectorname.insert(0,strip3)

# create empty data frame in pandas
df2 = pd.DataFrame()
#Add the first column to the empty dataframe.
df2['Sector']  = sectorname

#Now add the second column.
df2['Weight'] = sectorweight
#display(df2)

results = df.merge(df2, on = "Sector")
results.columns = ['Sector', 'IWD', 'IWF']
display(results)

Like I said, this works, but it isn't automated and its ham-handed way of getting there. Please help me to get better!


texts:
I've refactored my previous question finder, and added multi-tag support, and support for searching for keywords in the question titles. Essentially, what it does is find questions that could be answered based on the following parameters.

-- @MinimumQuestionVotes - The minimum amount of votes on a question.
-- @MaximumQuestionAnswers - The maximum amount of answers to a question.
-- @QuestionTags - The question's tags. Blank string is a wildcard.
-- @TitleContains - Words in the title. Blank string is a wildcard.

If you feel that any parameters aren't needed, or can be improved, just mention it. I'm open to any suggestions. Anyways, here's the code, and here's the SEDE query link.

-- User parameters for finding questions. Here is
-- a brief description of what each parameter does.

  -- @MinimumQuestionVotes - The minimum amount of votes on a question.
  -- @MaximumQuestionAnswers - The maximum amount of answers to a question.
  -- @QuestionTags - The question's tags. Blank string is a wildcard.
  -- @TitleContains - Words in the title. Blank string is a wildcard.
DECLARE @MinimumQuestionVotes INT = ##MinimumQuestionVotes:int?0##;
DECLARE @MaximumQuestionAnswers INT = ##MaximumQuestionAnswers:int?0##;
DECLARE @QuestionTags NVARCHAR(150) = ##QuestionTags##;
DECLARE @TitleContains NVARCHAR(250) = ##TitleContains##;

-- Final results are SELECTed based on the following
-- conditions.

  -- ClosedDate  IS EQUAL TO              null
  -- PostTypeId  IS EQUAL TO              question
  -- Score       GREATER THAN OR EQUAL TO @MinimimQuestionVotes
  -- AnswerCount LESS THAN OR EQUAL TO    @MaximumQuestionAnswers
  -- Tags        CONTAIN                  @QuestionTags
SELECT
    [Posts].Id AS [Post Link]
  , [Posts].OwnerUserId AS [User Link]
  , [Posts].Score
  , [Posts].Tags
  , [Posts].ViewCount
  , [Posts].AnswerCount
  FROM Posts WHERE
        [Posts].ClosedDate IS NULL
    AND [Posts].PostTypeId = 1
    AND [Posts].Score >= @MinimumQuestionVotes
    AND [Posts].AnswerCount <= @MaximumQuestionAnswers
    AND [Posts].Tags LIKE '%' + REPLACE(@QuestionTags, ' ', '%%') + '%'
    AND [Posts].Title LIKE '%' + REPLACE(@TitleContains, ' ', '%%') + '%'
  ORDER BY Score DESC;

Finally, here's some example input for the parameters. Do note, in the @QuestionTags and @TitleContains fields, you need to put single quotes, '', and space-separate each tag/word.

@MinimumQuestionVotes:   1
@MaximumQuestionAnswers: 0
@QuestionTags:           'python beginner'
@TitleContains:          'web'


texts: print array of previous smaller elements

texts:
I am solving interview question from here.

  Problem : Given an array, find the nearest smaller element G[i] for every element A[i] in the array such that the element has an index smaller than i.

  More formally, G[i] for an element A[i] = an element A[j] such that
      j is maximum possible and j < i and A[j] < A[i].
  Elements for which no smaller element exist, consider next smaller element as -1.

  Example: Input : A : [4, 5, 2, 10, 8] Return : [-1, 4, -1, 2, 2]

How can I optimise my solution:

def prev_smaller(arr):
    new_stack = []
    status = False
    for i in range(len(arr)):
        for j in range(i,-1,-1):
            if arr[i]>arr[j]:
                status = True
                new_stack.append(arr[j])
                break
        if not status:
            new_stack.append(-1)
        status = False
    return new_stack

assert prev_smaller([34,35,27,42,5,28,39,20,28]) == [-1,34,-1,27,-1,5,28,5,20]
assert prev_smaller([1]) == [-1]
assert prev_smaller([1,4,5,6]) ==[-1,1,4,5]


texts:
I learned javascript from some tutorials and as per advice, I thought of creating some task by my self. I wrote the following code and as I expected I saw the answer. If someone can assist me with how to code better? (The below code will create a table and update rows with input data which user enter)

// Javascript
var input = document.getElementById("input");
var btn   = document.getElementById("btn");
var table = document.getElementById("table");

btn.addEventListener('click', function(){
    var updateTask = input.value;
    // Now create row and add all the inputs value to a table
    var row = table.insertRow(-1);
    var cell1 = row.insertCell(0);
    var cell2 = row.insertCell(1);

    var allRows = table.querySelectorAll("tr");

    cell2.innerHTML = updateTask;
    // Update the Number/Order
    var rowLength = allRows.length;
    if (rowLength > 0){
        cell1.innerHTML = rowLength - 1;
    }
});

To Do List

    <input id="input" class="input" type="text" placeholder="Enter here!!!" onfocus="this.value=' ' ">
    <input id="btn" type="submit" value="ADD">

Number	Task


texts: Golang WebSocket communication between server and app

texts:
I've written a simple application that captures frames from my webcam and streams the frames via a websocket.

The stream implementation I've come up with feels weird. Networking is not my strong suit. I am wondering if the communication between the server and app can be written better.

type Stream struct {
    Frame []byte
}

func (s *Stream) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    conn, err := upgrader.Upgrade(w, r, nil)
    if err != nil {
            log.Fatal(err)
    }

    //This feels weird
    //I use a ticker that ticks with the fps of the camera
    //to send the individual frames to the browser

    ticker := time.NewTicker(time.Millisecond * 33)
    for _ = range ticker.C {
            if err = conn.WriteMessage(websocket.TextMessage, s.Frame); err != nil {
                    log.Fatal(err)
            }
    }
}

s := server.Stream{}
wg := sync.WaitGroup{}
wg.Add(2)
go func() {
        http.Handle("/", &server.TemplateHandler{Filename: "index.html"})
        http.Handle("/ws", &s)
        http.ListenAndServe(":8080", nil)
}()

go func() {
        err = camera.StartStreaming()
        checkErr(err)

        for {
                frame, err := camera.ReadFrame()
                checkErr(err)

                //Also weird, but hey it works
                s.Frame = []byte(base64.StdEncoding.EncodeToString(frame))
            }
}()

wg.Wait()


texts:
its a practice project from "Automate the Boring stuff with Python" book. i am an intermediate level Python programmer and i tried to solve this problem with less code as possible. This code will not take any wrong date into consideration eg: 29-02-2002 will not be selected because 2002 is not a leap year and only leap years have 29th of feb. i did not add code to also detect dates with months written in words, i could do that too but i want to keep things simple for now and i also did not use pyperclip module to detect dates from copied text to clipboard because i dont want to confuse any beginner who also want to learn from watching my code.
I want master programmers to review my code and if their is another way possible to detect dates then please post your solutions. Also i would appreciate any advice and positive criticism, so i know where i am standing right now and what i need to improve. Thanks. Code is as follows:
    import re

    def date_detector(text):
        date_pattern = re.compile('''
        ([12][0-9]|3[0-1]|0?[1-9])             # to detect days from 1 to 31
        ([./-])                                # to detect different separations
        (1[0-2]|0?[1-9])                       # to detect number of months
        ([./-])                                # to detect different seperations
        (2?1?[0-9][0-9][0-9])                  # to detect number of years from 1000-2999 years
         ''', re.VERBOSE)

        days = []
        months = []
        years = []
        dates = []
        for date in date_pattern.findall(text):
            days.append(int(date[0]))
            months.append(int(date[2]))
            years.append(int(date[4]))

        for num in range(len(days)):

        # appending dates in a list that dont need any filtering to detect wrong dates
            if months[num] not in (2, 4, 6, 9, 11):
                dates.append([days[num], months[num], years[num]])

        # detecting those dates with months that have only 30 days
            elif days[num] < 31 and months[num] in (4, 6, 9, 11):
                dates.append([days[num], months[num], years[num]])

        # filtering leap years with Feb months that have 29 days
            elif months[num] == 2 and days[num] == 29:
                if years[num] % 4 == 0:
                    if years[num] % 100 == 0:
                        if years[num] % 400 == 0:
                            dates.append([days[num], months[num], years[num]])
                    else:
                        dates.append([days[num], months[num], years[num]])

        # appending Feb dates that have less than 29 days
            elif months[num] == 2 and days[num] < 29:
                dates.append([days[num], months[num], years[num]])

        if len(dates) > 0:
            for date in dates:
                print(date)

data = '30-06-2012, 31-12-2012, 15-02-2002, 29-02-2004, 29-02-2002, 31-02-2004, 31-06-2012'

date_detector(data)
```


texts:
The following code uses scikit-learn to carry out K-means clustering where \$K = 4\$, on an example related to wine marketing from the book DataSmart. That book uses excel but I wanted to learn Python (including numPy and sciPy) so I implemented this example in that language (of course the K-means clustering is done by the scikit-learn package, I'm first interested in just getting the data in to my program and getting the answer out).

I'm new to Python so any advice on style or ways to write my code in a more idiomatic way would be appreciated.

The csv files needed (in the same directory as the program code) can be produced from downloading "Chapter 2" from the book link above and saving the first and second sheets of the resulting excel file as csv.

# -*- coding: utf-8 -*-
"""
A program to carry out Kmeans clustering where K=4
on data relating to wine marketing from book
"Data Smart: Using Data Science to Transform Information into Insight"

Requires csv input file OfferInfo.csv with headings
'Campaign', 'Varietal', 'Minimum Qty (kg)', 'Discount (%)', 'Origin', 'Past Peak'
and input file Transactions.csv with headings
'Customer Last Name', 'Offer #'
"""

#make more similar to Python 3
from __future__ import print_function, division, absolute_import, unicode_literals

#other stuff we need to import
import csv
import numpy as np
from sklearn.cluster import KMeans

#beginning of main program

#read in OfferInfo.csv
csvf = open('OfferInfo.csv','rU')
rows = csv.reader(csvf)
offer_sheet = [row for row in rows]
csvf.close()

#read in Transactions.csv
csvf = open('Transactions.csv','rU')
rows = csv.reader(csvf)
transaction_sheet = [row for row in rows]
csvf.close()

#first row of each spreadsheet is column headings, so we remove them
offer_sheet_data = offer_sheet[1:]
transaction_sheet_data = transaction_sheet[1:]

K=4 #four clusters
num_deals = len(offer_sheet_data) #assume listed offers are distinct

#find the sorted list of customer last names
customer_names = []
for row in transaction_sheet_data:
    customer_names.append(row[0])
customer_names = list(set(customer_names))
customer_names.sort()
num_customers = len(customer_names)

#create a num_deals x num_customers matrix of which customer took which deal
deal_customer_matrix = np.zeros((num_deals,num_customers))
for row in transaction_sheet_data:
    cust_number = customer_names.index(row[0])
    deal_number = int(row[1])
    deal_customer_matrix[deal_number-1,cust_number] = 1
customer_deal_matrix = deal_customer_matrix.transpose()

#initialize and carry out clustering
km = KMeans(n_clusters = K)
km.fit(customer_deal_matrix)

#find center of clusters
centers = km.cluster_centers_
centers[centers<0] = 0 #the minimization function may find very small negative numbers, we threshold them to 0
centers = centers.round(2)
print('\n--------Centers of the four different clusters--------')
print('Deal\t Cent1\t Cent2\t Cent3\t Cent4')
for i in range(num_deals):
    print(i+1,'\t',centers[0,i],'\t',centers[1,i],'\t',centers[2,i],'\t',centers[3,i])

#find which cluster each customer is in
prediction = km.predict(customer_deal_matrix)
print('\n--------Which cluster each customer is in--------')
print('{:<15}\t{}'.format('Customer','Cluster'))
for i in range(len(prediction)):
    print('{:<15}\t{}'.format(customer_names[i],prediction[i]+1))

#determine which deals are most often in each cluster
deal_cluster_matrix = np.zeros((num_deals,K),dtype=np.int)
print('\n-----How many of each deal involve a customer in each cluster-----')
print('Deal\t Clust1\t Clust2\t Clust3\t Clust4')
for i in range(deal_number):
    for j in range(cust_number):
        if deal_customer_matrix[i,j] == 1:
            deal_cluster_matrix[i,prediction[j]] += 1

for i in range(deal_number):
    print(i+1,'\t',end='')
    for j in range(K):
        print(deal_cluster_matrix[i,j],'\t',end='')
    print()
print()

print('The total distance of the solution found is',sum((km.transform(customer_deal_matrix)).min(axis=1)))


texts:
We would like to perform n-fold non-stratified cross-validation on a model, say an lm or glm.

We have a list of quality metrics we would like to evaluate, which we wrap in a function:

fn_rmse <- function(v_pred,v_true) sqrt(mean((v_true-v_pred)^2))
fn_mae <- function(v_pred,v_true) mean(abs(v_true-v_pred))
l_q <- list(fn_rmse, fn_mae, fn_smdape)
fn_q <- function(v_pred, v_true) {
  return(sapply(l_q, function(f)f(v_pred,v_true)))
}

Now we set up a cross-validation routine:

# Set up n-fold non-stratified cross-validation routines

# fn_fold: Run cross validation on a fold
# Input:
#   df_train = training data
#   df_test = testing data
#   mdl = model to test
#   fn_q = quality metrics to try
# Output: list of qualities for this fold
fn_fold <- function(df_train, df_test, mdl, fn_q) {
    mdl_fold <- update(mdl, data=df_train)
    v_predictions <- predict(mdl_fold, newdata=df_test, type="response")[[1]]
    fmla <- formula(mdl$call["formula"])
    s_regressand <- all.vars(fmla[[2]])
    v_quality <- sapply(v_predictions, fn_q, v_true=df_test[[s_regressand]])
    return(v_quality)
}

# fn_cv: Perform n-fold cross validation
# Input:
#   df = sample
#   mdl = model
#   n_folds = number of folds
#   fn_q = quality metrics to try
# Output: ([# of metrics] x n_folds)-matrix containing cross validation results
fn_cv <- function(df, mdl, n_folds, fn_q) {
    v_folds <- cut(seq(1,nrow(df)), breaks=n_folds, labels=FALSE)
    v_folds <- sample(v_folds)
    v_results <- sapply(1:n_folds,
        function(n_f) fn_fold(
            df[-which(v_folds==n_f,arr.ind=TRUE),],
            df[which(v_folds==n_f,arr.ind=TRUE),],
            mdl,
            fn_q)
        )
    return(v_results)
}

To run 10-fold cross-validation on a model, I call:

mtx_cv_results <- fn_cv(df=df_my_sample,
                        mdl=mdl_my_model,
                        n_folds=10,
                        fn_q=fn_q)

This feels a bit like reinventing the wheel. Can anything be done to make this code nicer?


texts: Add existing models to a relation using nested attributes

texts:
I needed to add existing models to a has-many relation using nested attributes so I overwrote thumbnails_attributes=:

class Gallery < ActiveRecord::Base
  has_many :thumbnails, dependent: :nullify

  accepts_nested_attributes_for :thumbnails

  def thumbnails_attributes=(thumbnails_attributes)
    ids = thumbnails_attributes.map { |t| t['id'] }
    (ids - thumbnail_ids).each do |id|
      thumbnail = Thumbnail.find id
      thumbnails << thumbnail
    end
    super(thumbnails_attributes)
  end
end

class Thumbnail < ActiveRecord::Base
  belongs_to :gallery
end

Given the following data (as fixture):

thumbnails.yml

one:
  gallery: one

two:
  gallery:

galleries.yml

one:
  name: MyString

I can now run:

galleries(:one).update(thumbnails_attributes: ['id' => thumbnails(:two).id])

and galleries(:one).thumbnails will return both thumbnails.

While it works pretty well, I wanted to ask for reviews and ways to improve/clean this code.

The only issue I have is with include? (which is used by assert_includes in my tests):

galleries(:one).update(thumbnails_attributes: ['id' => thumbnails(:two).id])
galleries(:one).thumbnails.include? thumbnails(:two) # false
galleries(:one).thumbnails                           # [#<Thumbnail one>, #<Thumbnail two>]
galleries(:one).thumbnails.include? thumbnails(:two) # true

It looks like I have to reload the relation for include? to work, but I haven't found a fix nor a way to properly test it.

galleries(:one).thumbnails.to_a.include? thumbnails(:two) works, include? doesn't appear to load the association.


texts:
I was asked to write a "Rock Paper Scissors" game as a practice exercise for class. I ended up with this code which works as intended. It's a one player vs. the computer game.

Overview:

Computer and player choose their 'weapons'
Decide the winner
Print the winner

Does anyone have any constructive feedback or tips on how I could improve this?

import random as r

#the possible moves in rock paper scissors
RPS = ["ROCK", "PAPER", "SCISSORS"]

def main():
    comp = computerMove()
    player = playerMove()
    winner, tie = calcWinner(comp, player)
    dispWinner(winner, tie)

#the move made by the computer
def computerMove():
   comp =  r.choice(RPS)
##   print(comp)
   return comp

#move made by the user
def playerMove():
    player = ""
    while not player in RPS:
            player = input("Choose your weapon(rock, paper, or scissors: ").upper()
##    print(player)
    return player

#determining a winner
def calcWinner(comp, player):
    print("The computer chooses {0}.".format(comp))
    winner = None
    tie = False

## if theres a tie
    if player == comp:
        print("No winner.... Restarting....\n")
        main()
        tie = True

## if the user wins
    elif ((comp == "ROCK") and (player == "PAPER")) or ((comp=="PAPER") and (player == "SCISSORS")) or ((comp=="SCISSORS") and (player == "ROCK")):
        winner = True
## if computer won
    else:
        winner = False

    return winner, tie

def dispWinner(winner, tie):
    if tie == True:
        print("")
    elif winner == True:
        print("You beat the computer!")
    elif winner == False:
        print("The computer beat you!")

main()


texts: Keep ABC size down for conditional string concatenation in Ruby

texts:
This method takes a comment object and displays it correctly depending on what columns are available. EVENT_TEXT and REASON_TEXT are hashes that map properties to more human readable strings, like so

EVENT_TEXT = { edit_item: 'edited an item' }
REASON_TEXT = { pickup_date: 'a pickup date change' }

The method that calls this one prepends this text with a username, so the text output expected is something along the lines of edited an item due to a pickup date change: the supplier is not ready yet.

  def comment_text(comment)
    text = ''
    if comment.event.present?
      text += EVENT_TEXT[comment.event.to_sym]
      if comment.reason.present?
        text += ' due to ' + REASON_TEXT[comment.reason.to_sym]
      end
    end
    text + ': ' + comment.text if comment.text.present?
  end

Which has a an Assignement Branch Condition size of 17, higher than Rubocop's default of 15. Is there a better way to concatenate strings conditionally in Ruby? I really don't feel like this method is doing too much.

Should I just increase Rubocop's threshhold? But even if I did, how would I handle a scenario like this with 10 conditions?


texts:
Microsoft Excel cells basically have two values: the formatted value that is shown to the user, and the actual value of the cell. These two values can be extremely different when the cell has certain formatting types such as date, datetime, currency, numbers with commas, etc.

I am writing a script that cleans up database data. It uses find, replace, RegEx, and other functions that act on strings of text. I need to be able to convert data into very specific formats so that my upload tool will accept the data. Therefore I need all of the cells in my Excel spreadsheets to be TEXT. I need the displayed values to match the actual values.

This is surprisingly hard to do in Excel. There is no paste special option that I could find that pastes what you see on your screen as text. "Paste Special -> Values" pastes, predictably, unformatted values. So if you try to "Paste Special -> Values" into cells formatted as TEXT, and they are coming from cells formatted as DATES, you might get something like this:

Old Cell: 4/7/2001

New Cell: 36988

If you paste normally, then the paste goes in as 4/7/2001, but in date format. So if I create a RegEx that looks for 4/7/2001 and changes it to 4/7/01, it will not work, because the underlying value is 36988.

Anyway, I discovered the Cell.Text property and wrote some code that converts all cells in a spreadsheet from formatted to text. But the code has two major problems:

1) It does not work on cells with more than 255 characters. This is because Cell.Text literally displays what the cell is actually showing to the user, and if the column is too narrow, it will display pound signs (########), and columns have a maximum width of 255 characters.

2) It is extremely slow. In my 1.5 million cell test sheet, code execution took 394 seconds.

I'm looking for ideas to optimize the code's speed, and to make the code work on all display value lengths.

Function ConvertAllCellsToText()

    ' Convert all cells from non-text to text
    ' This is helpful in a variety of ways
        ' Dates and datetimes can now be manipulated by replace, RegEx, etc., since their value is now 4/7/2001 instead of 36988
        ' Phone numbers and zip codes won't get leading zeros deleted if we try to add those

    StartNewTask "Converting all cells to text"

    ' first draft of the code, does not work as intended, dates get scrambled
    'Cells.Select
    'Selection.Copy
    'Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks _
        :=False, Transpose:=False
    'Application.CutCopyMode = False

    Dim MyRange As Range
    Set MyRange = ActiveSheet.UsedRange

    ' prevent Cell.Text from being ########
    MyRange.ColumnWidth = 255

    Dim i As Long
    Dim CellText As String
    For Each Cell In MyRange
        If Cell.Text <> Cell.Value And Len(Cell.Value) <= 255 Then
            CellText = Cell.Text
            Cell.NumberFormat = "@"
            Cell.Value = CellText
        End If

        i = i + 1
        If i Mod 2500 = 0 Then
            RefreshScreen
        End If
    Next

    MyRange.ColumnWidth = 20

    ' set all cells to Text format
    Cells.Select
    Selection.NumberFormat = "@"

End Function


texts:
I have retackled this problem using the help I received from here: Remove duplications from a Java String but this time using a LinkedHashSet since previosuly I was using a HashSet but the answer was out of order.

From this implementation my run time should be \$O(n)\$ correct?
Does anyone see any room where I can improve my code or some mistakes?

public static void main(String[] args){
    String test = "Banana";
    LinkedHashSet knownChars = new LinkedHashSet();
    StringBuilder noDups = new StringBuilder();

    for(Character c : test.toCharArray()){
        if(!knownChars.contains(c)){
            knownChars.add(c);
            noDups.append(c);
        }
    }
    System.out.println("No duplicate string is: " + noDups);
}


texts: Finding roughly matching genome sequences in Python dictionary

texts:
The purpose of my code here is to play a part in genome sequencing analysis, and while functional it takes days to run, so I am looking for any way I can improve speed. The input is up to 500 million lines long (making speed code efficiency important) and contains sequencing reads and corresponding info. Each read takes up 4 lines within the input file and looks something like this:

@A001             <-header
AAAAACCCCCCCCCCCC <-seq read (finalRead)
+
################# <-quality (trimmed_quality)

The portion of my code that is very slow takes a dictionary as input, which contains all of the data found within the input sequencing file and is in the form shown below:

duplexDict[umi] = {'header':header, 'seq':finalRead, 'qual':trimmed_quality}

In the first part of the code I am looking for pairs of sequences by checking for similar keys (termed umi in the code). The goal is to find keys that when converted to complement sequence are only different by a single letter. Then for each key if there is only one closely matching key, the associated dictionaries are retained. If there are no matches or more than one matching key, all of these keys should be ignored.

from Levenshtein import distance
deDuplexDict = {} # dict that will contain key pairs
finalList = [] # list to keep track of valid key pairs
for i in duplexDict: # dict with sequencing file info
    tempList = []
    for j in duplexDict:
        complement = str(Seq(j).complement()) # this is just finding complementary sequence
        if distance(i,complement) <= 1: # find similar umi/read seq pairs
            tempList.append(j) # make a list of all similar pairs
    # only keep a complementary pair if there are exactly two matching consensus reads
    if len(tempList) == 1:
        if i not in finalList and j not in finalList:
            finalList.append(i)
            finalList.append(j)

# only retain those dict values that are true pairs
for key in finalList:
    deDuplexDict[key] = duplexDict[key]

The second piece is designed to now collapse combine the sequences of two matching dictionary keys together and output to file. This is done by taking the complement of one of the sequences and then comparing each character position along the sequence strings. If anything doesn't match the character in a final string is just set to 'N' rather than the character found in the reads.

from itertools import combinations
prevScanned=[]
plus = '+'

# only pairs now exist, just search for them
for key1, key2 in combinations(deDuplexDict, 2):
    finalRead = ''
    complement = str(Seq(key2).complement()) # complement of second read sequence
    # if neither key has been analysed and they are a matching pair then use for consensus read
    if distance(key1, complement) <= 1 and key1 not in prevScanned and key2 not in prevScanned:
        prevScanned.extend([key1,key2]) # keep track of analyzed keys

        # convert to complementary matches
        refRead = deDuplexDict[key1]['seq']
        compRead = str(Seq(deDuplexDict[key2]['seq']).complement())

        # iterate through by locus and derive consensus
        for base in range(readLength):
            if refRead[base] == compRead[base]:
                finalRead += refRead[base]
            else:
                finalRead += 'N' # only perfect matches are permitted

        # output consensus and associated info
        target = open(final_output_file, 'a')
        target.write(deDuplexDict[umi]['header'] + '\n' + finalRead + '\n' + plus + '\n' + deDuplexDict[umi]['qual'] + '\n')
target.close()


texts: Extracting the location for a page with fewer conditionals

texts:
I've been given the job of trying to reduce the line count for a file that helps generate a PDF for a web app. The class itself is pdfHelper, and contains a number of methods used to generate a PDF such as getting the location, custom methods for formatting lowagie.text table cells etc.

I came across this particular method:

    public String getLocationToString(Page aPage) {
    if (aPage.getSiteDetail() != null || aPage.getLocationDescription() != null) {
        if (aPage.getSiteDetail() != null && aPage.getLocationDescription() == null) {
            return aPage.getSiteDetail();
        } else if (aPage.getSiteDetail() != null && aPage.getLocationDescription().getLocationLevel() == null) {
            return aPage.getSiteDetail();
        } else if (aPage.getSiteDetail() != null && aPage.getLocationDescription().getLocationLevel() != null) {
            return aPage.getSiteDetail().trim().toString() + "-" + aPage.getLocationDescription().getLocationLevel();
        } else {
            return "N/A";
        }
    } else {
        return "N/A";
    }
}

The method getSiteDetail() returns a String, getLocationDescription retrieves a LocationDescription object from the Page object. getLocationLevel() is a method for the LocationDEscription object and also returns a String.

Is there a way to shorten this method? I can't seem to figure out a way to reduce the amount of conditionals in this method (admittedly I'm not much of a programmer yet).


texts: Increasing size of video thumbnail on mouseover

texts:
This is a user JavaScript for YouTube. The point is to make the thumbnail bigger on mouseover. I just want some help making it better because I don't want it to waste resources with all the extensions I have and other scrips. I also added Iframes because I have another extension that makes a preview of the video in the thumbnail. I also use Opera so make sure it works on Opera.

/*! settings begin */
//Makes the name of the video apear bellow the image if false (false/true)
var WideSideBar = true;
/*! settings end */

//Get the sidebar were the thumbnails are and the main
var videos = document.getElementById("watch7-sidebar");
var main = document.getElementById("watch7-main");
//Make the SideBar wider
if (WideSideBar) {
    main.style.width = "1117px";
    videos.style.width = "472px";
}

videos.addEventListener("mouseover", function (a) {
    if (a.target.tagName == "IMG") {
        //Make image bigger
        a.target.style.width = "292px";
        a.target.style.height = "163px";
        a.target.style.WebkitTransition = 'width 0.5s, height 0.5s';
        a.target.parentNode.parentNode.parentNode.style.width = "292px";
        a.target.parentNode.parentNode.parentNode.style.height = "163px";
        a.target.parentNode.parentNode.parentNode.style.WebkitTransition = 'width 0.5s, height 0.5s';

        //Get a higher resolution image of the thumbnail
        var video_id = a.target.parentNode.parentNode.parentNode.parentNode.parentNode.href.split('v=')[1];
        var ampersandPosition = video_id.indexOf('&');
        if (ampersandPosition != -1) {
            video_id = video_id.substring(0, ampersandPosition);
        }
        a.target.src = "http://img.youtube.com/vi/" + video_id + "/0.jpg";

    } else if (a.target.tagName == "IFRAME") {

        //Make Iframe AKA:(Thumbnail preview) bigger
        a.target.parentNode.style.WebkitTransition = 'width 0.5s, height 0.5s';
        a.target.parentNode.style.width = "292px";
        a.target.parentNode.style.height = "163px";
    }
}, false);

videos.addEventListener("mouseout", function (a) {
    if (a.target.tagName == "IMG") {

        //Make image normal size.
        a.target.style.width = "120px";
        a.target.style.height = "67px";
        a.target.parentNode.parentNode.parentNode.style.width = "120px";
        a.target.parentNode.parentNode.parentNode.style.height = "67px";
    } else if (a.target.tagName == "IFRAME") {
        //Make Iframe AKA:(Thumbnail preview) normal size
        a.target.parentNode.style.width = "120px";
        a.target.parentNode.style.height = "67px";
    }
}, false);


texts:
There are 3 search fields, and at least one must be filled out. When the search button is clicked the search fields are checked, if none are filled in show the error window with the error message for this error. If at least one field is filled in get a code based on the search criteria. If the code was not successfully retrieved show same error window but with the error message for this error.

The code I have works fine for this except that it looks verbose and I am repeated the code in the else statements except for the unique error messages. How can this be improved?

private void searchButton_Click(object sender, RoutedEventArgs e)
{
    searchCriteria.Clear();
    searchCriteria.Add(townSearchTextbox.Text);
    searchCriteria.Add(countySearchTextbox.Text);
    searchCriteria.Add(postcodeSearchTextbox.Text);

    if (isValidSearchCriteria(searchCriteria))
    {
        formatSearchCriteria(searchCriteria);
        set.updateWOEID(searchCriteria);
        if (set.WOEID.Length > 0)
        {
            // continue
        }
        else
        {
            // display any errors in the error window
            Error errorWindow = new Error();
            errorWindow.Show();
            errorWindow.errorMessage.Text = "Error: Could not retrieve WOEID, please try again.";
        }
    }
    else
    {
        // display any errors in the error window
        Error errorWindow = new Error();
        errorWindow.Show();
        errorWindow.errorMessage.Text = "Error: Please provide one or more search criteria.";
    }
}


texts: Theano code for 2-layer neural net for mnist

texts:
I did not attach the data, but it is the standard mnist in CSV format. I am new to theano, and I would like comments on the implementation of this code, as well as suggestions on better code-writing.

import pandas
import numpy
import theano
import theano.tensor as T
rng = numpy.random

def accuracy(predictions, start=0):
    count = 0.0
    for i in range(len(predictions)):
        if predictions[i] == train_data["label"][start+i]:
            count = count + 1.0

    accuracy = count/len(predictions)
    print "--- Accuracy value is " + str('%0.5f' %accuracy)
    return accuracy

train_data = pandas.read_csv("Data/train.csv")
test_data = pandas.read_csv("Data/test.csv")

predictors = []
for i in range(784):
    string = "pixel" + str(i)
    predictors.append(string)

#training rate
tr_rate = 0.01
# number of training steps
training_steps = 10001
# number of neurons in hidden layer
nhidden = 400

feats = train_data.shape[1]-1 #removing the label from the count
num_classes = 10 #10 digits

# Declare Theano symbolic variables
x = T.matrix("x")
y = T.ivector("y")

rng = numpy.random.RandomState(0)
# Weights and biases for input to hidden layer
wh = theano.shared(rng.normal(0, 0.1, (feats, nhidden)), borrow = True)
bh = theano.shared(numpy.full(nhidden, 0.0), borrow = True)

# The weights and biases from hidden to output layer
wy = theano.shared(rng.normal(0,0.1, (nhidden, num_classes)), borrow = True)
by = theano.shared(numpy.full(num_classes, 0.0), borrow = True)

hid =  T.nnet.sigmoid(T.dot(x, wh) + bh)         # Activation function of hidden layer

# Construct Theano expression graph
#sigma = 1 / (1 + T.exp(-T.dot(x, w) - b))       # Probability that target = 1
sigma = T.nnet.softmax(T.dot(hid,wy) + by)
prediction = T.argmax(sigma, axis=1)             # The class with highest probability

# Cross-entropy loss function
xent = -T.mean(T.log(sigma)[T.arange(y.shape[0]), y])

parameters = [wh, bh, wy, by]  # this line defines all parameters for each layer

# Regularisation
cost = xent.mean() + 0.25 * ( (wh ** 2).sum() + (wy ** 2).sum() ) #+ 0.01 * ( abs(wh).sum() + abs(wy).sum() )
gparameters = [T.grad(cost, param) for param in parameters]    # Compute the gradient of the cost

# Compile
trainf = theano.function(
          inputs=[x,y],
          outputs=[cost, prediction],
          updates=[(param, param - tr_rate * gparam) for param, gparam in zip(parameters, gparameters)],
          allow_input_downcast=True)

predict = theano.function(inputs=[x], outputs=prediction)

hiddenImg = theano.function(
              inputs=[x],
              outputs=[hid] )

numpy.random.seed(0)
step = 500
maxV = train_data[predictors].shape[0] - step
# Train
for i in range(training_steps):
    start = numpy.random.randint(0, maxV)
    output = trainf(train_data[predictors][start:start+step], train_data["label"][start:start+step])
    if i%1000 == 0 :
        print "Gone through " + str(i) + " steps"
        accuracy(output[1], start)

output = trainf(train_data[predictors], train_data["label"])
accuracy(output[1])


texts:
I'm making a personal logger and so far I have come up with 3 solutions.
I have them simplified in this example to highlight my dilemma. I want to know the pros and cons of my approaches and ways to make them better. Also should I be flushing the stream as often as possible and why?

TempLog

This version will be created and destroyed on every log. Endline forced on destruction.

class TempLog
{
public:
    std::ostream& log()
    {
        return std::cout;
    }
    ~TempLog()
    {
        std::cout << "\n";
    }
};

SingletonLog_ver_1

If the endline is forced in this manner using the same ostream from outside the logger will result in bad output. Also it feels somewhat dishonest but due to my lack of experience I don't trust my feeling.

class SingletonLog_ver_1
{
public:
    static SingletonLog_ver_1 instance()
    {
        static SingletonLog_ver_1 instance;
        return instance;
    }

    std::ostream& log()
    {
        static bool firstLine = true;
        if (firstLine)
            firstLine = false;
        else
            std::cout << "\n";

        return std::cout;
    }

private:
    SingletonLog_ver_1() {};
};

SingletonLog_ver_2

Now while I like what is happening here more than the previous two examples it seems like the least elegant code. Is there another more elegant way to achieve this kind of functionality?

class SingletonLog_ver_2
{
public:
    static SingletonLog_ver_2 instance()
    {
        static SingletonLog_ver_2 instance;
        return instance;
    }

    std::pair<std::ostream&, std::unique_ptr<TempObj_NewLine>> log()
    {
        return std::pair<std::ostream&, std::unique_ptr<TempObj_NewLine>>(std::cout, std::make_unique<TempObj_NewLine>());
    }

private:
    SingletonLog_ver_2() {};
};

TempObj_NewLine

class TempObj_NewLine
{
public:
    TempObj_NewLine() {};
    ~TempObj_NewLine()
    {
        std::cout << "\n";
    }
};

Use of the 3 loggers:

int main()
{
    TempLog().log() << "first log!" << " first log, same line.";
    TempLog().log() << "second log." << "second line.";

    SingletonLog_ver_2::instance().log().first << "first line singleton log!"<< " first line, again.";
    SingletonLog_ver_2::instance().log().first << "second line singleton log!";

    SingletonLog_ver_1::instance().log() << "first line singleton - hacky endline." << " first line, again.";
    SingletonLog_ver_1::instance().log() << "second line singleton - hacky endline";

    std::cin.get();
}


texts: Elegant way to "construct" kind-of enum values in go

texts:
I'm quite new to Go and would like to improve my code. I have a representation for access levels in Go like this:

type AccessLevelValue int

const (
    GuestPermissions     AccessLevelValue = 10
    ReporterPermissions  AccessLevelValue = 20
    DeveloperPermissions AccessLevelValue = 30
    MasterPermissions    AccessLevelValue = 40
    OwnerPermission      AccessLevelValue = 50
)

What I want is a "constructor" for AccessLevelValue to which I can pass a numeric value from {10,20,30,40,50} and that returns me the according AccessLevelValue. This is my current implementation, but it seems not so nice to me:

func int2AccessLevel(accessLevel int) *AccessLevelValue {
    if accessLevel == 10 { return AccessLevel(GuestPermissions)}
    if accessLevel == 20 { return AccessLevel(ReporterPermissions)}
    if accessLevel == 30 { return AccessLevel(DeveloperPermissions)}
    if accessLevel == 40 { return AccessLevel(MasterPermissions)}
    if accessLevel == 50 { return AccessLevel(OwnerPermission)}
    return nil
}

func AccessLevel(v AccessLevelValue) *AccessLevelValue {
    p := new(AccessLevelValue)
    *p = v
    return p
}

Is there any cleaner implementation that doesn't need the if statements and returns the same result?


texts:
This is a working program I made to roll dice a specified number of times, show how many times each number (2-12) occurred, and the percentage of the total rolls that each number got.

I'm trying to make improvements to the efficiency of my code. I've already found one, as noted beside int[] results on line 15. Are there any tools I could have used to make this code more efficient in terms of performance, or that could have saved me time? (Such as ArrayList, case/switch etc)

import java.util.Random;
import java.util.Scanner;
import java.text.DecimalFormat;
public class Dice {
    public static void main(String[] args){
        DecimalFormat df = new DecimalFormat("#.#"); //how to display percentage of results
        Scanner sc = new Scanner(System.in);
        System.out.println("How many times would you like to roll the dice?");
        int rolls = sc.nextInt();
        int[] counts = new int[rolls]; //creates an array of user-specified length (rolls)
        sc.close();
        for(int i = 0; i < counts.length; i++){ //populates said array with each roll of the dice
            counts[i] = diceRoll();
        }
        int[] results = counters(counts); /*creates a new array with the returned array that counters(counts) gave us.
                                        Seems redundant, but it is noticeably faster at high numbers to iterate through this array
                                        than it is to access the counters method 3 times in the loop below (by using counters(counts) instead of results).*/

        System.out.println("Roll\t\tCount\t\tFrequency"); //Headers for the results table
        for(int i = 0; i < results.length; i++){ //iterates through our results array, prints how many times each number was rolled, along with its percentage of the total rolls
            System.out.println((i+2)/*to start the table at 2, since rolling a 0 is not possible*/+"\t\t" + (results[i]) + "\t\t"+df.format(100.0*(results[i])/counts.length)+ "%");
        }
    }
    public static int diceRoll(){ //rolls 2 6-sided dice and returns their sum
        Random rand = new Random();
        int dice1 = rand.nextInt(6)+1;
        int dice2 = rand.nextInt(6)+1;
        int roll = dice1+dice2;
        return roll;
    }
    public static int[] counters(int[] arr){ //keeps track of how many times each possible number is rolled, puts the results into an array and returns the array
        int c2=0;
        int c3=0;
        int c4=0;
        int c5=0;
        int c6=0;
        int c7=0;
        int c8=0;
        int c9=0;
        int c10=0;
        int c11=0;
        int c12=0;
        for(int i = 0; i < arr.length; i++){
            if(arr[i]==2)
                c2++;
            else if(arr[i]==3)
                c3++;
            else if(arr[i]==4)
                c4++;
            else if(arr[i]==5)
                c5++;
            else if(arr[i]==6)
                c6++;
            else if(arr[i]==7)
                c7++;
            else if(arr[i]==8)
                c8++;
            else if(arr[i]==9)
                c9++;
            else if(arr[i]==10)
                c10++;
            else if(arr[i]==11)
                c11++;
            else if(arr[i]==12)
                c12++;
        }
        int[] rollCounts = new int[] {c2, c3, c4, c5, c6, c7, c8, c9, c10, c11, c12};
        return rollCounts;
    }
}


texts:
For homework, I have to code a program in Python that effectively simulates a bank ATM.

print('Welcome to Northen Frock Bank ATM')
restart=('Y')
chances = 3
balance = 67.14
while chances >= 0:
    pin = int(input('Please Enter You 4 Digit Pin: '))
    if pin == (1234):
        print('You entered you pin Correctly\n')
        while restart not in ('n','NO','no','N'):
            print('Please Press 1 For Your Balance\n')
            print('Please Press 2 To Make a Withdrawl\n')
            print('Please Press 3 To Pay in\n')
            print('Please Press 4 To Return Card\n')
            option = int(input('What Would you like to choose?'))
            if option == 1:
                print('Your Balance is Â£',balance,'\n')
                restart = input('Would You you like to go back? ')
                if restart in ('n','NO','no','N'):
                    print('Thank You')
                    break
            elif option == 2:
                option2 = ('y')
                withdrawl = float(input('How Much Would you like to
withdraw? \nÂ£10/Â£20/Â£40/Â£60/Â£80/Â£100 for other enter 1: '))
                if withdrawl in [10, 20, 40, 60, 80, 100]:
                    balance = balance - withdrawl
                    print ('\nYour Balance is now Â£',balance)
                    restart = input('Would You you like to go back? ')
                    if restart in ('n','NO','no','N'):
                        print('Thank You')
                        break
                elif withdrawl != [10, 20, 40, 60, 80, 100]:
                    print('Invalid Amount, Please Re-try\n')
                    restart = ('y')
                elif withdrawl == 1:
                    withdrawl = float(input('Please Enter Desired amount:'))

            elif option == 3:
                Pay_in = float(input('How Much Would You Like To Pay In? '))
                balance = balance + Pay_in
                print ('\nYour Balance is now Â£',balance)
                restart = input('Would You you like to go back? ')
                if restart in ('n','NO','no','N'):
                    print('Thank You')
                    break
            elif option == 4:
                print('Please wait whilst your card is Returned...\n')
                print('Thank you for you service')
                break
            else:
                print('Please Enter a correct number. \n')
                restart = ('y')
    elif pin != ('1234'):
        print('Incorrect Password')
        chances = chances - 1
        if chances == 0:
            print('\nNo more tries')
            break

This program basically does what it is designed to do, but I want to learn how
to use the def() and do it this way.


texts:
How can I improve the date format function. I have written in javascript (ES5/ES6)? This code is just a part of an exercise.

function formatDate() {
        let bookingDate = new Date();
        let dayNumber = bookingDate.getDay();
        let date = bookingDate.getDate();
        let month = bookingDate.getMonth();
        let year = bookingDate.getFullYear();

        return ` ${getWeekday(dayNumber)}, ${monthName(
      month
    )}  ${date},  ${year}`;

        function getWeekday(dayNumber) {
            let day = bookingDate.getDay();

            let DayName = [
                "Sunday",
                "Monday",
                "Tuesday",
                "Wednesday",
                "Thursday",
                "Friday",
                "Saturday"
            ];

            return DayName[dayNumber];
        }

        function monthName(month) {
            let monthNames = [
                "January",
                "February",
                "March",
                "April",
                "May",
                "June",
                "July",
                "August",
                "September",
                "October",
                "November",
                "December"
            ];

            return monthNames[month];
        }
    }


texts:
I am trying to teach myself haskell by means of project Euler

i sovled problem #9 by means of this code which is on the brute force side I was wondering if there is a more elegant/haskell/efficient solution

let project_euler_9 = [(a,b,c) | c <- [1..500], b <- [1..c], a <- [1..b], a^2 + b^2 == c^2, a+b+c==1000 ]


texts:
I'm trying to do the Reverse Groups Challenge on CodeEval:

  Given a list of numbers and a positive integer k, reverse the elements of the list, k items at a time. If the number of elements is not a multiple of k, then the remaining items in the end should be left as is.

My logic is functional, but when I try to submit my solution is says that it timed out at 10 seconds. How can I improve the speed here?

File.open(ARGV.first).readlines.each do |line|
  values, k = line.split(';')
  k = k.to_i
  values = values.split(',')
  current = values.shift(k)
  str = ""
  while current.count % k == 0 do
    current.reverse_each{ |val| str += "#{val},"}
    current = values.shift(k)
  end
  str << "#{current.join(',')}"
  puts str
end

Other attempt:

open(ARGV[0]).readlines.each do |line|
  values, k = line.chomp.split(';')
  k = k.to_i
  values = values.split(',')
  current = values.shift(k)
  str = ""
  while current.count % k == 0 do
    current.reverse_each{ |val| str << "#{val},"}
    current = values.shift(k)
  end
  str << "#{current.join(',')}"
  puts str
end


texts: x86 assembly [fasm] code which takes a number and adds 155 then shifts right

texts:
The following program takes a user-inputted integer and adds 155, shifts it to the right twice, and prints the output after the addition, then after the shift. It is written in fasm x86 assembly and uses the stdcall convention:

format PE Console
entry main
include 'macro/import32.inc'

; Read-Only data
section '.rdata' data readable
msg db 'Enter a number and we will do some math to it: ', 0
msg2 db 13,10, 'The final result is: %d', 0, 13, 10
p db 'pause>nul', 0
msg3 db 13, 10, '%d + 155 is %d', 0
msg4 db 'You entered: %d', 0
section '.data' data readable writeable
frmspec db '%d'
number dd 0

section '.code' code readable executable

main:
push ebp
mov ebp, esp
; no local vars needed
push msg
call [printf]
sub esp, 4 ; clean stack
push number
push frmspec
call [scanf]
sub esp, 8 ; clean stack
push [number]
push msg4
call [printf]
sub esp, 8 ; clean stack

mov eax, [number]
add eax, 155
push eax
push [number]
push msg3
call [printf]
pop ecx
pop eax
pop ecx

shr eax, 2
mov dword [number], eax
push [number]
push msg2
call [printf]
sub esp, 8
mov esp, ebp
pop ebp
push p
call [system]
sub esp, 4
push 0
call [exit]

;imports
section '.idata' import data readable
library msvcrt, 'msvcrt.dll'
import msvcrt, \
printf, 'printf', \
system, 'system', \
scanf, 'scanf', \
exit, 'exit'


texts:
I am taking Stanford's Introduction to Databases Self-Paced online course. I have gone through the videos in the SQL mini-course, and I am having trouble completing the exercises.

The following is the question from the SQL Movie-Rating Query Exercises, Question 7:

  For each movie that has at least one rating, find the highest number
  of stars that movie received. Return the movie title and number of
  stars. Sort by movie title.

The database can be found here.

My answer to this question is as follows:

SELECT distinct Movie.title, Rate.stars
FROM Movie, Rating,
(SELECT * FROM Rating R1 WHERE not exists (SELECT mID FROM Rating R2 WHERE R1.stars < R2.stars and R1.mID = R2.mID)) as Rate
WHERE Movie.mID = Rate.mID and Rate.stars = Rating.stars
order by Movie.title;

This seems like a very tortured query, and it seems to me like I am missing some important concepts. Can someone help me refactor this query?


texts:
I am trying to figure out a more efficient way of converting money into change. The code below is what i have written so far. It works perfectly but i know the code can be improved. Is there a more efficient way of writing this code?

public class Problem2 {

    public static int quarters(int total)
    {
        return total/25;
    }

    public static int dimes(int total)
    {
        return (total - quarters(total)*25) /10;
    }

    public static int nickels(int total)
    {
        return (total - (quarters(total)*25) - (dimes(total)*10)) / 5;
    }

    public static int pennies(int total)
    {
        return total - (quarters(total)*25) - (dimes(total)*10) - (nickels(total)*5);
    }

    public static void main(String[] args) {
        Scanner keyboard = new Scanner(System.in);
        String total;
        int intTotal;
        System.out.println("Hi, this application will take your number and will display the minimum amount of"
                + "coins necessary to represent the dollar amount");
        System.out.println("Please enter in the total amount of dollars and cents:");
        total = keyboard.next();

        total = total.replace(".","");
        intTotal = Integer.parseInt(total);

        System.out.println(quarters(intTotal) + " Quarters,"
                          +dimes(intTotal) + " Dimes,"
                          +nickels(intTotal) + " Nickels,"
                          +pennies(intTotal) + " Pennies");

    }

}


texts: Approximating pi/4 using Wallis Product - SICP exercise 1.31

texts:
From SICP

  The sum procedure is only the simplest of a vast number of similar
  abstractions that can be captured as higher-order procedures. Write an
  analagous procedure called product that returns the product of the
  values of a function at points over a given range.  ...  "Also
  use product to compute approximations to  pi using the formula
  pi/4 = 2*4*4*6*6*8*.../3*3*5*5*7*7*...

Here is a procedure called product:

(define (product term a next b)
  (if (> a b)
      1
      (* (term a) (product term (next a) next b))))

I have made two solutions:

Here is the first solution. This one navigates through the series by a period of 2 (E.G. if n is 1, it will evaluate 2*4/3*3). I did this because I wanted to find a pattern from the series and what I got was 2*4 and 4*6 and 6*8, and the denominators 3*3 and 5*5 and 7*7.

(define (pi-product2 n)
    (define (inc x) (+ x 1))
    (define (numer-term x)
      (* (* 2.0 x)
         (+ (* 2.0 x)
            2.0)))
    (define (denom-term x)
      (square (+
               (* 2.0 x)
               1.0)))
    (define (term x)
      (/ (numer-term x) (denom-term x)))
    (product term 1.0 inc n))

Here is my second solution. I just recently thought of how to evaluate them by each term (E.G. if n is 1, it will evaluate 2/3).

  (define (pi-product n)
    (define (inc x) (+ x 1))
    (define (term x)
      (if (even? x)
          (/ (+ x 2.0) (+ x 1.0))
          (/ (+ x 1.0) (+ x 2.0))))
    (product term 1.0 inc n))

Is my first solution still a correct representation of the formula even if it evaluates two terms on each approximation? Or is it the first solution with the correct representation?
I would think my first solution, though very ugly, is at least as twice as fast as the second solution. Is that right? Is the performance difference huge?

Whitch of my two solution is better? How can I improve them?


texts: Using interface for generated functions or just create plain functions

texts:
I was wondering which code might be better. I'm abstracting away the rote code as follows:

type InvokerArgs = any[]
interface InvokerFunctions {
  /**
  * send message to colleague
  */
  sendMessage: (smid: number, message: string) => InvokerArgs
  notifyOnline: (smid: number, firstTime: boolean) => InvokerArgs
}

var invokerFunctions = <InvokerFunctions> [
  'sendMessage',
  'notifyOnline'
].reduce((functions, functionName) => {
  functions[functionName] = (...args) => {
    return [functionName].concat(args)
  }
  return functions
}, {})

export = invokerFunctions

/*** Example ***/
import invokers = require('invokers')
invoke(invokers.sendMessage(46257, 'Hi, partner!'))

This seems pretty nice since the core functionality is concise, simple and easy to read. The interface makes it so you can see what each function takes.

/*** Alternative implementation ***/

function baseInvokerArgs2 (args) {
  // TypeScript in VS 2013 doesn't let me create an interface for
  // `Function` with the `name` property
  return [(<string> (<any>args.callee).name)].concat(_.toArray(args))
}

/**
* send message to colleague
*/
export function sendMessage (smid: number, message: string): InvokerArgs {
  return baseInvokerArgs2(arguments)
}

export function notifyOnline (smid: number, firstTime: boolean): InvokerArgs {
  return baseInvokerArgs2(arguments)
}

/*** Example ***/
import invokers = require('invokers')
invoke(invokers.sendMessage(46257, 'Hi, partner!'))

This alternative solution seems like it's more verbose. But I don't need to repeat the function name like in the previous version. But I do need to repeat the return statement with its function call.

At this time I think I like the first solution more, but maybe just because it is more clever. But it doesn't take advantage of some of TypeScripts stronger typing by generating the functions on the fly, so if I miss copying the name over the function won't actually be there even though the interface is or vice versa.

If it was pure JavaScript with no IntelliSense I would probably just go with the first solution with comments about the parameters.


texts:
I've written a simple scraper that parses HTML using BeautifulSoup and collects the data (schedule of sports events), then clubs them together in a list of dicts.

The code works just fine, but the way I process the data is pretty horrible IMO. I use an if...else to parse the data selectively, because the output of the dicts are alternative, that is {venue, result (if available)}, and time of match, and team information.

So basically, is there a better, more reliable way to parse the data?

import requests
import re
from BeautifulSoup import BeautifulSoup
from datetime import datetime
class cricket(object):

    def getMatches(self, url):
        """ Scrape the given url for match schedule """

        headers = {'Accept':'text/css,*/*;q=0.1',
        'Accept-Charset':'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
        'Accept-Encoding':'gzip,deflate,sdch',
        'Accept-Language':'en-US,en;q=0.8',
        'User-Agent':'Mozilla/5 (Solaris 10) Gecko'}

        page = requests.get(url, headers = headers)
        page_content = page.content
        soup = BeautifulSoup(page_content)

        result = soup.find('div', attrs={'class':'bElementBox'})
        tags = result.findChildren('tr')

        match_type_list = ['TEST', 'ODI', 'T20']
        match_info = []

        for elem in range(1,len(tags)):
            dict_ = {}
            x = tags[elem].getText()
            x = x.replace(r'&nbsp;', '')

            if 'Venue' in x:
                for a in match_type_list:
                    if a in x:
                        match_type = a

                x = x.replace('Venue', '')
                if 'Result' in x:
                    x = x.replace('Result', '')
                    x = x.split(': ')
                    # print x
                    venue = x[1]
                    result = x[2]
                    dict_.update({'venue':venue,'result':result,
                        'match_type':match_type})
                else:
                    x = x.split(': ')
                    venue = x[1]
                    dict_.update({'venue':venue})

            else:
                match = re.search(r'\b[AP]M', x)
                date_time = x[0:match.end()]
                date_time = date_time.replace(',','')[4:]
                teams = x[match.end():].split('vs')
                home_team = teams[0].strip()
                away_team = teams[1].strip()
                # print date_time, home_team, away_team

                time_obj = datetime.strptime(date_time, '%b %d %Y %I:%M %p')
                timings = time_obj.strftime('%Y-%m-%dT%H:%MZ')
                dict_.update({'home_team':home_team,
                    'away_team':away_team,
                    'timings':timings
                })

            match_info.append(dict_)

        # print match_info

        final_list = []     # final list of dicts that we need

        for i in range(0, len(match_info), 2):
            final_list.append(dict(match_info[i].items() +
                match_info[i+1].items()))

        # for i in final_list:
        #     print i
        # print final_list

if __name__ == '__main__':
    url = 'http://icc-cricket.yahoo.net/match_zone/series/fixtures.php?seriesCode=ENG_WI_2012' # change seriesCode in URL for different series.
    #url = 'http://localhost:6543/lhost/static/icc_cricket.html'
    c = cricket()
    c.getMatches(url)


texts: Is there a more efficient way of executing multiple SQL prepared statements at once?

texts:
I'm updating more than one table at once and, as you will see, it uses a lot of similar looking code. Is there a more efficient way of doing this, perhaps some way of combining the queries?

For context: A user submits a small form containing information about their age, gender and favorite genre of games.

String sex, age;
String[] checkboxes = request.getParameterValues("genre");

sex = request.getParameter("sex");
age = request.getParameter("age");

try(Connection con = ds.getConnection()){
    PreparedStatement updateSex = null;
    PreparedStatement updateAge = null;
    PreparedStatement updateGenre = null;

    String updateSexString = "UPDATE sex SET count=count+1 WHERE name=?";
    String updateAgeString = "UPDATE age SET count=count+1 WHERE id=?";
    String updateGenreString = "";

    for(int i = 0; i < checkboxes.length; i++){

        updateGenreString = "UPDATE genre SET count=count+1 WHERE id=?";
        updateGenre = con.prepareStatement(updateGenreString);
        updateGenre.setString(i+1, checkboxes[i].toString());
        updateGenre.executeUpdate();
        updateGenre.close();
    }

    updateSex = con.prepareStatement(updateSexString);
    updateSex.setString(1, sex);
    updateSex.executeUpdate();
    updateSex.close();

    updateAge = con.prepareStatement(updateAgeString);
    updateAge.setString(1, age);
    updateAge.executeUpdate();
    updateAge.close();

} catch(SQLException e){
    e.printStackTrace();
}


texts:
I have a Product entity and a ProductCategory entity. Given the Product entity, my application checks the database via a repository to see if the ProductCategory already exists. If the category is not found in the database, then the application should persist it.

Currently, I have this code in the service layer as follows:

public class ProductCategoryMapper implements DataMapper<ProductCategory> {

    @NonNull
    private final ProductCategoryRepository categoryRepository;

    @Override
    public void map(Product product) {
        ProductCategory category = productCategoryRepository.findByProductTagAndMake(product);

        if (category == null) {
           category = productCategoryRepository.save(new ProductCategory(product.getTag(), product.getMake()));
        }
        product.setProductCategory(category);
    }
}

But should this method be in the Product class as:

public void mapCategory(); // or perhaps setCategory();

This seems to be better OO but then the Product entity would have a reference to the repository (which I think is bad from a design / DDD perspective b/c the entity knows about repositories). Should the code stay in the service layer, be moved to the entity, or is there some other option?


texts:
I'd like to check the following process relating to the uploading of data by FTP, after the data is read from a database.

I have a stored procedure that returns data in the following pipe delimited format in a single column. It typically returns around 300 rows, each row has about 300 bytes.

1234|||||PROPERTY|||MARKET INDEX|||||ADDRESS|||||1||||GBP|||||||20110930||||||||C|OFFICE|F||

The application then submits that data by FTP, typically <100k overall.

Here's the mechanism used, restricted to .NET 3.5.

// database method returns dataset/datareader, converted and passed to method as
// IEnumerable<string> data

string ftpServerIP = "5.4.3.2:21";
string targetFileName = "mann.txt";
string username = "manfred";
string password = "*******";

Uri uri = new Uri(String.Format("ftp://{0}/{1}", ftpServerIP, targetFileName));
FtpWebRequest reqFTP = (FtpWebRequest)FtpWebRequest.Create(uri);
reqFTP.Credentials = new NetworkCredential(username, password);
reqFTP.Method = WebRequestMethods.Ftp.UploadFile;
reqFTP.KeepAlive = false;
reqFTP.UsePassive = false;

MemoryStream stIn = new MemoryStream();
using (StreamWriter sw = new StreamWriter(stIn))
{
    foreach (string item in data)
    {
        sw.WriteLine(item);
    }

    sw.Flush();

    using (Stream stOut = reqFTP.GetRequestStream())
    {
        stOut.Write(stIn.GetBuffer(), 0, (int)stIn.Length);
    }
}

FtpWebResponse response = (FtpWebResponse)reqFTP.GetResponse();
response.Close();

I'd like to check whether the stream and ftp processing looks ok, or whether there are other stream types to consider (BufferedStream?), or closure/disposal steps, etc...


texts: Creating a string from command line arguments

texts:
I am fairly new to C and am just overcoming my fear of pointers and segmentation faults. C is a strange language to me because I come from more object oriented languages like JavaScript and Java.

When I code in C, everything feels all "one-liney" to me. I'm used to things, for example in Java, being neatly spread out, everything with it's own clean section in the code.

When I see C(ha) that I write, everything is all in one place and nothing has it's own area.

Here is an example piece of code that I recently wrote. It takes all the command line arguments and puts them all into one string with each word separated by a space (dismiss any and all spelling errors):

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    if(argc == 1) { // if there are too few arguments
        printf("ERROR: Expected atleast 1 argument\n");
        return 0;
    }
    int size = 0;
    while(argv[size]) { // gets the amount of arguments besides the first argument
        size++;
    }
    size--;
    if(size == 1) { // if there is only 1 word
        printf("%s\n", argv[1]);
        return 0;
    }
    int i;
    int v = strlen(argv[1]) + strlen(argv[2]); // for allocating memroy

    char *str = (char *)malloc(v);
    strcat(str, argv[1]); // putting the string together
    strcat(str, " ");
    strcat(str, argv[2]);
    strcat(str, " ");
    for(i = 3; i <= size; i++) {
        str = (char *)realloc(str, (v + strlen(argv[i]))); // reallocates enough memory
        strcat(str, argv[i]);
        strcat(str, " ");
    }
    printf("%s\n", str);
    return 0;
}

One thing that I read somewhere about C involving convention is that C should be broken up into smaller bits of code (functions?). Is this true, and what else should I work on?


texts: Get the highest role of a user across all the organizations he is member of

texts:
I have a function that looks like this:

def getMaxUserLevelByUserId(userId: Muid)(
      implicit connection: Connection): Option[OrganizationMembership.Level.Value] = {
    val organizationMemberships = getAllByUserId(userId)

    if (organizationMemberships.exists(_.level == OrganizationMembership.Level.Admin)) {
      Some(OrganizationMembership.Level.Admin)
    } else if (organizationMemberships.exists(_.level == OrganizationMembership.Level.Member)) {
      Some(OrganizationMembership.Level.Member)
    } else if (organizationMemberships.exists(_.level == OrganizationMembership.Level.Guest)) {
      Some(OrganizationMembership.Level.Guest)
    } else {
      None
    }
  }

The gets the highest role of a user across all the organizations he is member of. As you can see, the function is pretty straightforward and works. However, it has a complexity of O(n3). I was wondering if there is a way to get it to O(n) or at least a bit lower?


texts: Find the max. difference between two array elements a[j] and a[i] such that j > i

texts:
The source of the problem is here.

  Given an array of integer elements, return the maximum difference of any pair of
  numbers such that the larger number of the pair occurs at a higher index
  than the smaller.

Here is my implementation in Java.

package algo.mindiff;

public class Main {

    // http://www.careercup.com/question?id=5185822661804032
    public static void main(String[] args) {
        System.out.println(minDiff(new int[] { 10, 1, 12, 3, 4, 28, 1 }));
    }

    private static int minDiff(int[] array) {
        int diff = 0;
        int result = 0;
        for (int i = 0; i < array.length;) {
            int maxIndex = maxIndex(array, i);
            int minIndex = minIndex(array, i, maxIndex);
            diff = array[maxIndex] - array[minIndex];
            if (result < diff)
                result = diff;
            i = maxIndex + 1;
        }
        return result;
    }

    private static int minIndex(int[] array, int start, int end) {
        int min = start;
        for (int i = start + 1; i < end; i++) {
            if (array[i] < array[min]) {
                min = i;
            }
        }
        return min;
    }

    private static int maxIndex(int[] array, int start) {
        int max = start;
        for (int i = start + 1; i < array.length; i++) {
            if (array[i] >= array[max]) {
                max = i;
            }
        }
        return max;
    }

}

And here is a brief summary of my algorithm:

I find the maximum element in the given array. Its index is maxIndex.
Then I find the minimum element within the range [0, maxIndex). Its index is minIndex.
If the difference result I calculated last time is less than the difference diff between array[maxIndex] and array[minIndex], I'll just assign the new value to result.
I do steps 1 - 3 for the other elements of the array starting with maxIndex + 1.

If my algorithm doesn't work on any input data you might have in mind, please let me know.


texts:
I've read other solutions about solving the 10th problem for Project Euler (finding the sum of all prime numbers under 2,000,000) but I wanted to try it on my own using the Sieve of Eratosthenes method. It runs quickly and it seems to work for all of my test increments (under 10, first 1000 primes) but it doesn't seem to work for the full 2,000,000 range. I was wondering if anyone could help point out the problem in my code. I know it's not the prettiest/most efficient code, so sorry.

#include <iostream>

using namespace std;

int main(int argc, char const *argv[])
{
    // upper limit & sum
    int sum = 0;
    int upLimit = 2000000;

    // array of isPrime check for integers
    bool isPrime[upLimit+1];

    // initialize
    for (int i=2; i <= upLimit; i++) {
        isPrime[i] = true;
    }

    for (int i=2; i<=upLimit; i++) {
        if (isPrime[i]) {
            // add to sum
            sum += i;

            // mark all multiples
            int j = i;
            while (j <= upLimit) {
                isPrime[j] = false;
                j += i;
            }
        }
    }
    cout << sum << endl;
}


texts: What is the most optimal way to concat XML files as one large string/text file?

texts:
Python3 - Recursive XML Files to Large Text File
I have a recursive function that looks through a directory (there are 6000+ files).
For each XML file found - the XML is converted to a string and appended to an array.
At the end of the recursion - All these arrays are joined to return a (very) large string.
The error I am getting is MemoryError - How can I optimise my code?
import os
import xml.etree.ElementTree as ET

# Cycle through a directory and concat files together - recursive dive.
# NOTE:
# Have a folder called 'PLC 2360' in the immediate directory.
# The output is a huge text file

path = 'PLC 2360'

big_text = []

def big_file(path):
    try:
        for i in next(os.walk(path))[2]:    # For each XML file read and convert to string
            xml = ET.parse(path + '/' + i).getroot()
            big_text.append(ET.tostring(xml, encoding='unicode'))    # Append string of XML
    except:
        print('Error on', path)

    if next(os.walk(path))[1]:    # If folders exist - cycle through
        for i in next(os.walk(path))[1]:
            big_file(path + '/' + i)    # Enter next recursion layer

    return ''.join(str(i) for i in big_text)    # Return one big string

lis = big_file(path)
print(len(lis))
print(lis[:500])
```


texts: Multiple lines of code in if statement without curly braces

texts:
I have used multiple lines of code in one if condition without using curly braces, which is executing properly. So what is the main necessity of using curly braces in if statements for multiple lines?

 if (ggcCompareGrid.TopLevelTable != null)
            foreach (GridRow row in ggcCompareGrid.TopLevelTable.Rows)
            {
                if (row.DisplayElement.ParentGroup != null)
                    if (row.DisplayElement.ParentGroup.Category != null)
                        if ((int)row.DisplayElement.ParentGroup.Category == key && row.Record == null)
                        {
                            row.DisplayElement.ParentGroup.IsExpanded = false;
                            break;
                        }
            }


texts: Categorizing gene sequences read from a CSV file

texts:
I am relatively new to programming and would love to get some feedback on the following section of my code.

class Gene:
    def __init__(self, gene_symbol, gene_id):
        # gene_symbol represents the abbreviated name of the gene (string)
        # Example: 'RHO' (short for 'Rhodopsin')
        self.gene_symbol = gene_symbol

        # gene_id represents an accession ID for the gene (string)
        # Example: 'NM_000539.3'
        self.gene_id = gene_id

        # is_valid_ functions check if the type of the gene_id is valid
        # and returns True or False
        # Assume that the types of the gene_id are mutually exclusive
        if is_valid_refseq(gene_id):
            # Example of a valid refseq: 'NM_000539.3'
            self.gene_id_type = REFSEQ

        elif is_valid_ensembl_gene(gene_id):
            # Example of a valid ensembl_gene: 'ENSG00000163914'
            self.gene_id_type = ENSEMBL_GENE

        elif is_valid_ensembl_transcript(gene_id):
            # Example of a valid ensembl_transcript: 'ENST00000296271'
            self.gene_id_type = ENSEMBL_TRANSCRIPT

        else:
            raise InvalidGeneIDError("Invalid gene_id: {}".format(gene_id))

refseqs = []
ensembl_genes = []
ensembl_transcripts = []

with open(csv_gene_list, 'r', newline='') as csv_input:

    reader = csv.reader(csv_input, delimiter=',')
    next(reader)
    for row in reader:
        row_gene_symbol = row[0]
        row_gene_id = row[1]

        row_gene = Gene(row_gene_symbol, row_gene_id)

        if row_gene.gene_id_type == REFSEQ:
            refseqs.append(row_gene)

        elif row_gene.gene_id_type == ENSEMBL_GENE:
            ensembl_genes.append(row_gene)

        elif row_gene.gene_id_type == ENSEMBL_TRANSCRIPT:
            ensembl_transcripts.append(row_gene)

        else:
            # What do I do here?
            raise AssertionError('Unrecognized gene_id_type: {}'.format(
                    row_gene.gene_id_type))

Assume gene_symbol and gene_id are strings and the is_valid_ functions are implemented elsewhere in my code.

I would like to ask if implementing these multiple if-else statements is the right approach. I feel like my code might get cluttered if I add more of these conditional expressions down the line.

Another question is on the very last else statement. What do I do here in a case like this? If the code works as intended, then it shouldn't reach the last else statement because an InvalidGeneIDError should've been raised when initializing the Gene object. One solution is to omit this part and change elif row_gene.gene_id_type == ENSEMBL_TRANSCRIPT: to else:, but I would much rather explicitly state the specific check to help convey the intentions of my code.

My main priority as I move forward is to focus on developing good coding style/habits and to write readable code. I would also like to learn/discover new tools/tricks that Python has to offer. Any suggestions and help would be greatly appreciated.


texts:
Today I was trying GSON after a while and went to the YQL console to run this query:

select * from yahoo.finance.xchange where pair in ("EURUSD","GBPUSD","INRUSD","JPYUSD")

and get back this JSON:

{
 "query": {
  "count": 4,
  "created": "2014-12-11T18:41:23Z",
  "lang": "en-US",
  "results": {
   "rate": [
    {
     "id": "EURUSD",
     "Name": "EUR to USD",
     "Rate": "1.238",
     "Date": "12/11/2014",
     "Time": "1:41pm",
     "Ask": "1.2382",
     "Bid": "1.2378"
    },
    {
     "id": "GBPUSD",
     "Name": "GBP to USD",
     "Rate": "1.5705",
     "Date": "12/11/2014",
     "Time": "1:41pm",
     "Ask": "1.5708",
     "Bid": "1.5703"
    },
    {
     "id": "INRUSD",
     "Name": "INR to USD",
     "Rate": "0.016",
     "Date": "12/11/2014",
     "Time": "1:40pm",
     "Ask": "0.016",
     "Bid": "0.016"
    },
    {
     "id": "JPYUSD",
     "Name": "JPY to USD",
     "Rate": "0.0084",
     "Date": "12/11/2014",
     "Time": "1:41pm",
     "Ask": "0.0084",
     "Bid": "0.0084"
    }
   ]
  }
 }
}

To parse it I wrote this code which works fine:

Wrapper.java

public class Wrapper {
    private Query query;

    public Query getQuery() { return query; }
}

Query.java

public class Query {
    private int count;
    private Date created;
    private String lang;
    private Result results;

    public int getCount() { return count; }
    public Date getCreated() { return created; }
    public String getLang() { return lang; }
    public Result getResults() { return results; }

}

Result.java

public class Result {
    private Rate[] rate;

    public Rate[] getRate() { return rate; }
}

Rate.java

public class Rate {
    private String id;
    private String Name;
    private Double Rate;
    private String Date;
    private String Time;
    private Double Ask;
    private Double Bid;

    public String getId() { return id; }
    public String getName() { return Name; }
    public Double getRate() { return Rate; }
    public String getDate() { return Date; }
    public String getTime() { return Time; }
    public Double getAsk() { return Ask; }
    public Double getBid() { return Bid; }
}

I created separate classes and then in a main class with a main method created the Gson object and did this:

Gson gson = new GsonBuilder().create();
Wrapper wrapper = gson.fromJson(jsonString, Wrapper.class);

But what differences would there be if I had the above classes as nested classes in the Wrapper.java class? Or is creating them separately and then putting them in a package more preferable?

What are the differences in terms of speed, efficiency, design, or anything else anyone could advise on?