texts:
This is my JavaScript function to get a break time taken. I have two times (time_start and time_end). And with these two times, I have to get my break time. (break time hours are 12:00-13:00 and 17:45-18:15). The code below works but it contains a lot of nested if and else. Is there any method to reduce them?

function timesheet(day,week_type,start_time,end_time){
        this.day = day;
        this.week_type = week_type;
        this.start_time = convertTime(start_time);
        this.start_time = (!isNaN(this.start_time)?this.start_time:0);
        this.end_time = convertTime(end_time);
        this.end_time = (!isNaN(this.end_time)?this.end_time:0);
        this.work_start = convertTime("9:00");
        this.work_end = convertTime("17:45");
        this.break_1_start = convertTime("12:00");
        this.break_1_end = convertTime("13:00");
        this.break_2_end = convertTime("18:15");
        //休憩時間を作業の開始時間と終業時間の次第で下の関数からゲットします。

        this.break_time = function(){
            /**
             *この関数が取った休憩時間を渡します。
            */
            if(this.start_time>this.end_time){
                return 0;
            }
            else{
                //if strart time is less than or equal to 12
                if(this.start_time<=this.break_1_start){
                    if(this.end_time<=this.break_1_start){
                        return 0;
                    }
                    else if(this.end_time<this.break_1_end){
                        return (0+(this.end_time - this.break_1_start));
                    }
                    else{
                        if(this.end_time<this.work_end){
                            return 1;
                        }
                        else if(this.end_time<this.break_2_end){
                            return (1+(this.end_time - this.work_end));

                        }
                        else{
                            return 1.5;
                        }
                    }
                }
                else if(this.start_time<this.break_1_end){
                    if(this.end_time<this.break_1_end){
                        return (this.end_time - this.start_time);
                    }
                    else {
                        if(this.end_time<this.work_end){
                            return (this.break_1_end - this.start_time);
                        }
                        else if(this.end_time<this.break_2_end){
                            let break_one = this.break_1_end - this.start_time;
                            let break_two = this.end_time - this.work_end;

                            return (break_one+break_two);
                        }
                        else{
                            let break_one = this.break_1_end - this.start_time;
                            let break_two = this.break_2_end - this.work_end;
                            return (break_two+break_one);
                        }
                    }
                }
                else if(this.start_time<this.work_end){
                    if (this.end_time<this.work_end){
                        return 0;
                    }
                    else{
                        if (this.end_time<this.break_2_end){
                            return (this.end_time-this.work_end);
                        }
                        else{
                            return(0.5);
                        }
                    }
                }
                else{
                    if(this.start_time<this.break_2_end){
                        return (this.break_2_end-this.start_time);
                    }
                    else{
                        return 0;
                    }
                }
            }
}


texts: Recursively walking a tree to build up a new one

texts:
Could this be better factored as a reduce?

/**
 * recursively build a nested Backbone.Model or Backbone.Collection
 * from a deep JSON structure.
 *
 * undefined for regex values
 */
exports.modelify = function modelify (data) {

    if (_.isArray(data)) {
        return new Backbone.Collection(_.map(data, modelify));
    }

    else if (_.isObject(data) && !_.isDate(data)) {

        var m = util.mapo(data, function(val, key) {
            return [key, modelify(val)];
        });

        return new Backbone.Model(m);
    }
    else {
        // data is a primitive or a date
        return data;
    }
}

var mapo = exports.mapo = _.compose(_.object, _.map);


texts: Find the longest rectilinear path in a matrix with descending entries

texts:
Problem (Skiing):

  Each number represents the elevation of that area of the mountain.

  From each area (i.e. box) in the grid, you can go north, south, east,
  west - but only if the elevation of the area you are going into is
  less than the one you are in (i.e. you can only ski downhill).

  You can start anywhere on the map and you are looking for a starting
  point with the longest possible path down as measured by the number of
  boxes you visit.

  And if there are several paths down of the same length, you want to
  take the one with the steepest vertical drop, i.e. the largest
  difference between your starting elevation and your ending elevation.

  Goal:

  Find the longest path following these rules. Longest path means the maximum number of nodes where we are always going down.

  If we have many longest paths with equal distance (number of nodes), then we should get the one with the maximum drop (value of 1st
  node - value of the last node).

The aim is to run this code for a 1000 x 1000 matrix. As for now, running for just a 4 x 4, it's taking minutes. Any idea on how to reduce the complexity regarding time and space?

myMatrix = [[4, 5, 2],[1, 1, 6],[8, 7, 5]]
def getAllValidSkiingPathsFrom(myMatrix):
    dctOfMatrix = {}
    for row in range(len(myMatrix)):
        for column in range(len(myMatrix[0])):
            currPoint = (column, row)
            dctOfMatrix[currPoint] = myMatrix[row][column]

    lstIndicesOfAllMatrixPoints = list(dctOfMatrix.keys())
    setAllPossiblePaths = set()

    from itertools import permutations
    for pathCandidate in permutations(lstIndicesOfAllMatrixPoints):
        lstPossiblePath = []
        prevIndexTuple = pathCandidate[0]
        lstPossiblePath.append(prevIndexTuple)
        for currIndexTuple in pathCandidate[1:]:
            if abs(currIndexTuple[0]-prevIndexTuple[0]) + abs(currIndexTuple[1]-prevIndexTuple[1]) > 1:
                break # current path indices not allowed in path (no diagonals or jumps)
            else:
                if dctOfMatrix[currIndexTuple] >= dctOfMatrix[prevIndexTuple]:
                    break # only "down" is allowed for "skiing"
                else:
                    lstPossiblePath.append(currIndexTuple)
                    prevIndexTuple = currIndexTuple
        if len(lstPossiblePath) > 1 and tuple(lstPossiblePath) not in setAllPossiblePaths:
            setAllPossiblePaths.add(tuple(lstPossiblePath))

    return setAllPossiblePaths, dctOfMatrix

setAllPossiblePaths, dctOfMatrix = getAllValidSkiingPathsFrom(myMatrix)

printedPath = []
bestPath = []
for path in setAllPossiblePaths:
    for point in path:
        printedPath.append(dctOfMatrix[point])
    if len(printedPath) > len(bestPath): # Looking for the path with a maximum distance
        bestPath = printedPath
    if len(printedPath) == len(bestPath): # If we have more than one path with a maximum distance we look for the drop
        if (printedPath[0]-printedPath[-1]) > (bestPath[0]-bestPath[-1]):
            bestPath = printedPath
    printedPath = []

print("Path -->", bestPath)
print("Distance -->", len(bestPath))
print("Drop -->", bestPath[0]-bestPath[-1])

Sample input matrix:

4 5 2
1 1 6
8 7 5

Output:

Path --> [8, 7, 1]
Distance --> 3
Drop --> 7

Explanation:

The longest path would be 8-7-1 as it has the maximum distance = 3.

There are other paths with the same maximum distance which are 8-7-5 and 5-4-1. However, 8-7-1 has the maximum drop = 7 (8-1).


texts: Two variants of a db_create_resources(…) function

texts:
After reading noobs, I'm interested in possible way to improve readability of the following short snippet.

a struct pictdb_file is a small file system containing a metadata array and a file descriptor fpdb.
the function allocates the resources, and forwards initialization to db_create_logic.

How can I improve the readability of this snippet ? Please comment about: the syntax, white-lines, comments, function extracting, etc.

Also, which version do you prefer and why ?

version 1:

enum error_codes
db_create_resources(const char* db_filename, struct pictdb_file* db_file)
{
    //allocate metadata
    db_file->metadata = calloc(db_file->header.max_files, sizeof(pict_metadata));

    if (db_file->metadata == NULL) {
        return ERR_OUT_OF_MEMORY;
    }

    //open fpdb
    db_file->fpdb = fopen(db_filename, "wb");

    if (db_file->fpdb == NULL) {
        free(db_file->metadata);
        db_file->metadata = NULL;
        return ERR_IO;
    }

    //forward
    enum error_codes e;
    e = db_create_logic(db_filename, db_file);

    //release resources
    free(db_file->metadata);
    db_file->metadata = NULL;
    db_close(db_file);

    return e;
}

version 2: (comment and blank-lines changes)

enum error_codes
db_create_resources(const char* db_filename, struct pictdb_file* db_file)
{
    db_file->metadata = calloc(db_file->header.max_files, sizeof(pict_metadata));
    if (db_file->metadata == NULL) {
        return ERR_OUT_OF_MEMORY;
    }

    db_file->fpdb = fopen(db_filename, "wb");
    if (db_file->fpdb == NULL) {
        free(db_file->metadata);
        db_file->metadata = NULL;
        return ERR_IO;
    }

    enum error_codes e;
    e = db_create_logic(db_filename, db_file);

    free(db_file->metadata);
    db_file->metadata = NULL;
    db_close(db_file);

    return e;
}


texts: Constantly checking for activity in a page then log out if no activity

texts:
function executeQuery() {
  $.ajax({
    url: '../include/activity.php',
    success: function(data) {
      // do something with the return value here if you like
      if(data.error == true){
           window.location.href = "login.php";
      }

    }
  });
  setTimeout(executeQuery, 3000); // you could choose not to continue on failure...
}

$(document).ready(function() {
  // run the first time; all subsequent calls will take care of themselves
  setTimeout(executeQuery, 3000);
});

In activity.php

if (isset($_SESSION['LAST_ACTIVITY']) && (time() - $_SESSION['LAST_ACTIVITY'] > 300)) {
    // last request was more than 5 minutes ago
    session_unset();     // unset $_SESSION variable for the run-time
    session_destroy();   // destroy session data in storage
echo json_encode(array(
    'error' => true
));
}
$_SESSION['LAST_ACTIVITY'] = time(); // update last activity time stamp

if (!isset($_SESSION['CREATED'])) {
    $_SESSION['CREATED'] = time();
    echo json_encode(array(
        'error' => false
    ));
} else if (time() - $_SESSION['CREATED'] > 300) {
    // session started more than 5 minutes ago
    session_regenerate_id(true);    // change session ID for the current session an invalidate old session ID
    $_SESSION['CREATED'] = time();  // update creation time
    echo json_encode(array(
        'error' => false
    ));
}

This is how I am checking for activity of user using ajax and php.
I have doubts if this is acceptable in real life environment.
I would like to have suggestion on how to implement this cleanly , correctly easily.


texts: square the elements of a sorted list and give the output in sorted order

texts:
Given a sorted list of integers, square the elements and give the
  output in sorted order.

  For example, given [-9, -2, 0, 2, 3], return [0, 4, 4, 9, 81].

My solution 1:

const square = el => el * el;
const sortAsc = (a, b) => a - b;
const sortSquare = list => list
  .map(square)
  .sort(sortAsc);

console.log(sortSquare([-9, -2, 0, 2, 3]));

My solution 2:

const sortSquare2 = list => {
  list.sort((a, b) => Math.abs(a) - Math.abs(b));
  for (let i = 0; i < list.length; i++) {
    list[i] = Math.pow(list[i], 2);
  }
  return list;
};

console.log(sortSquare2([-9, -2, 0, 2, 3]));

Is there a faster solution? I have the feeling you can do something with the fact that the list is sorted to begin with. But I can't think of a good one.


texts:
Can someone please optimisations for my implementation of min heap.Its performance is poor compared to the heapq module in python. My use case for the MinHeap class is 'N' insertions and extractions where N > 1,000,000.

import math

class MinHeap:
    def __init__(self, arr):
        self.__items = arr
        self.build_heap()

    def size(self):
        """
        Returns the size of heap.
        """
        return len(self.__items)

    def items(self):
        """
        Returns the elements in the heap.
        """
        return self.__items

    def build_heap(self):
        """
        Builds heap using the list of elements.
        """
        for node_number in xrange(int(len(self.__items) / 2), 0, -1):
            self.heapify(node_number)

    def heapify(self, node_number):
        """
        Ensure that node follows heap property.
        """
        # return if leave node
        if node_number > int(len(self.__items) / 2):
            return

        node = self.__items[node_number-1]
        left_child = self.__items[(2 * node_number)-1] if (((2 * node_number)-1) < len(self.__items)) else None

        right_child = self.__items[(2 * node_number + 1)-1] if (((2 * node_number + 1)-1) < len(self.__items)) else None

        min_node = node

        if left_child != None and right_child != None:
            min_node = min(node, left_child, right_child)
        elif left_child != None :
            min_node = min(node, left_child)
        elif right_child != None :
            min_node = min(node, right_child)

        if min_node == node:
            return
        elif left_child!=None and min_node == left_child:

            self.__items[node_number - 1], self.__items[(2 * node_number)-1] = self.__items[(2 * node_number)-1], self.__items[node_number - 1]
            self.heapify(2 * node_number)
        elif right_child!=None and min_node == right_child:

            self.__items[node_number - 1], self.__items[(2 * node_number + 1)-1] = self.__items[(2 * node_number + 1)-1], self.__items[node_number - 1]
            self.heapify(2 * node_number + 1)

    def extract_min(self):
        """
        Returns the minimum element.
        """
        length = len(self.__items)
        if length == 0:
            return
        self.__items[0], self.__items[length-1] = self.__items[length-1], self.__items[0]
        min_element =  self.__items.pop()
        self.heapify(1);
        return min_element

    def insert(self, num):
        """
        Inserts a new element in the heap.
        """
        self.__items.append(num)
        current_node = len(self.__items)
        parent_node = int(current_node / 2)
        while current_node > 1:
            min_node = min(self.__items[current_node-1], self.__items[parent_node-1])
            if min_node == self.__items[parent_node-1]:
                break
            self.__items[current_node-1], self.__items[parent_node-1] = self.__items[parent_node-1], self.__items[current_node-1]
            current_node = parent_node
            parent_node = int(current_node / 2)


texts: New to programming. How can I write this Rock, Paper, Scissors code better?

texts:
I am practicing w/ Python and wanted to code a simple Rock, Paper, Scissors game w/o any user input. I used a list to hold the values and randomly pulled them with a function. I used the the return value from the first function to print the response in the second function, along with the printed results. What are some alternative methods to achieve this same functionality and make this more efficient. Thank you.

# Internal Rock, Paper Scissors Game  >>>>>>>>>>>>>>>>>>>>>>>>>

import random

# List string values for the game

r_p_s = [
    'Rock', 'Paper', 'Scissors'
]

# Function to return random values from List
def game():
    return random.choice(r_p_s) + " vs. " + random.choice(r_p_s)

result = game()  # Returned Value

# Function to print out the results of the returned value with a response

def response():
    print("Roll: " + "\n" + result)

    if "Scissors vs. Paper" == result or \
       "Paper vs. Scissors" == result:
        print("\nScissors beats Paper")

    elif "Scissors vs. Rock" == result or \
         "Rock vs. Scissors" == result:
        print("\nRock beats Scissors!")

    elif "Rock vs. Paper" == result or \
         "Paper vs. Rock" == result:
        print("\nPaper beats Rock!")

    else:
        print("\nDraw! Roll again.")

response()


texts:
In a web game there is a ranking of the top points games played of the month:

// QUERY
SELECT * FROM game
WHERE date >= date_sub(CURDATE(),INTERVAL 1 MONTH)
ORDER BY points DESC limit 0, 20

But, in that case one player can be the top 1,2,3... so I have changed the query to add a distinct, and then only appears one game for player (the game with max points):

// Query that filters player names
SELECT g.*
FROM game g
INNER JOIN ( SELECT playerName, MAX(points) AS MaxPoints
             FROM game
             GROUP BY playerName
           ) groupGame
        ON g.playerName = groupGame.playerName AND g.points = groupGame.MaxPoints
WHERE date >= date_sub(CURDATE(),INTERVAL 1 MONTH)
ORDER BY points DESC limit 0, 20

The query works OK but the second query is much slower than the first. Is there any way to optimize it?


texts: Comments on SASS from CSS for a SASS beginner

texts:
I've leaped into SASS and am loving it. I'm asking for comments here on whether my SASS is well-formatted.

Here's the original CSS based on Zurb's Foundation Pagination styles, but adapted for use with Laravel's built-in pagination:

div.pagination ul { display: block; height: 24px; margin-left: -5px; }
div.pagination ul li { float: left; display: block; height: 24px; color: #999; font-size: 14px; margin-left: 5px; }
div.pagination ul li a { display: block; padding: 1px 7px 1px; color: #555; font-weight: normal}
div.pagination ul li:hover a,
div.pagination li a:focus { background: $bgltblue; }
div.pagination ul li.disabled a { cursor: default; color: #999; }
div.pagination ul li.disabled:hover a,
div.pagination li.disabled a:focus { background: transparent; }
div.pagination ul li.active a { background: $headblue; color: white; font-weight: normal; cursor: default; }
div.pagination ul li.active a:hover,
div.pagination li.active a:focus { background: $headblue; }

Here's my manual conversion to SASS:

div.pagination {
  ul {
    display: block; height: 24px; margin-left: -5px;
    li {
      float: left; display: block; height: 24px; color: #999; font-size: 14px; margin-left: 5px;
      a {
        display: block; padding: 1px 7px 1px; color: #555; font-weight: normal;
      }
      &:hover a {
        background: $bgltblue;
      }
      &.disabled {
        a {
          cursor: default; color: #999;
        }
        &:hover a {
          background: transparent;
        }
      }
      &.active {
        a {
          background: $headblue; color: white; font-weight: normal; cursor: default;
          &:hover {
            background: $headblue;
          }
        }
      }
    }
  }
  li {
    a:focus {
      background: $bgltblue;
    }
    &.disabled {
      a:focus {
        background: transparent;
      }
    }
    &.active {
      a:focus {
        background: $headblue;
      }
    }
  }
}

I'd like to know if you think I have reduced it correctly, and where I might make improvements. I plan to replace the colours with vars already.


texts:
For my mathematical research, I am to deal with very large numbers and I know that the standard C types are not going to stand a chance against them due to their limited capacity, so I decided to write a super calculator in C language in hope that I won't have to resort to a super computer.

I started with an add function that I want to be reviewed for bugs if any, and possible improvement in performance and general programming aspects. Note that, on my system, the program was able to add a number consisting of 200 million digits of 9 to the same number in 0.98 seconds, which is pretty impressive, but still, if there is anything in performance that can be enhanced, let me know. That is because the multiplication function is going to be dependent on this function, and things will get slower because multiplication is repeated addition.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

char *add(char *x, char *y);

int main(void)
{
    clock_t t1 = clock();
    /* ............................................................................................................................. */
    size_t nd = 200 * 1000 * 1000; // 2 hundred million digits
    char *x = (char *)malloc(nd + 1);
    char *y = (char *)malloc(nd + 1);
    if (!x || !y) {
        fputs("error: memory allocation failed.\n", stderr);
        goto dismantle;
    }
    /* assign x and y */
    for (size_t i = 0; i < nd; i++) {
        x[i] = '9';
        y[i] = '9';
    }
    /* NUL-terminate the arrays */
    x[nd] = '\0';
    y[nd] = '\0';
    /* add x and y */
    char *z = add(x, y);
    if (z) {
        //printf("x + y = %s\n", z);
        free(z);
    }
dismantle:
    if (x) free(x);
    if (y) free(y);
    /* ............................................................................................................................. */
    clock_t t2 = clock();
    fprintf(stderr, "time elapsed: %.4f\n", (double)(t2 - t1) / CLOCKS_PER_SEC);
    fprintf(stderr, "Press any key to continue . . . ");
    getchar();
    return 0;
}

void ReverseArray(char *buf)
{
    char tmp, *ptr = buf + strlen(buf) - 1;
    while (buf < ptr) {
        tmp = *buf;
        *buf++ = *ptr;
        *ptr-- = tmp;
    }
}

char *add(char *x, char *y)
{
    /* store number of digits in size_t variables*/
    size_t xlen = strlen(x);
    size_t ylen = strlen(y);
    /* essential variables */
    char *z = NULL;
    int r = -1;
    int val, var = (xlen >= ylen);
    size_t n = 0;
    /* pointers to x and y */
    char *p1, *p2;
    char *t1, *t2;
    /* assign pointers according to the value of var */
    if (var) {
        p1 = x + xlen - 1;  t1 = x;
        p2 = y + ylen - 1;  t2 = y;
        z = (char *)malloc(xlen + 2);
    }
    else {
        p1 = y + ylen - 1;  t1 = y;
        p2 = x + xlen - 1;  t2 = x;
        z = (char *)malloc(ylen + 2);
    }
    /* check for allocation failure */
    if (!z) {
        fputs("error: memory allocation failed.\n", stderr);
        return NULL;
    }
    /* stage 1 */
    while (p2 >= t2)
    {
        if (r == -1) {
            val = (*p1-- - '0') + (*p2-- - '0');
        }
        else {
            val = (*p1-- - '0') + (*p2-- - '0') + 1;
        }
        if (val > 9) {
            r = val - 10;
            z[n++] = r + '0';
        }
        else {
            z[n++] = val + '0';
            r = -1;
        }
    }
    /* stage 2 */
    while (p1 >= t1)
    {
        if (r == -1) {
            z[n++] = *p1--;
        }
        else {
            val = (*p1-- - '0') + 1;
            if (val > 9) {
                r = val - 10;
                z[n++] = r + '0';
            }
            else {
                z[n++] = val + '0';
                r = -1;
            }
        }
    }
    /* stage 3 */
    if (r != -1) {
        z[n++] = 1 + '0';
    }
    z[n] = '\0';
    ReverseArray(z);
    return z;
}


texts: Return a 404 when a resource is not found in Spring Boot

texts:
I wrote this code in order to return a http status of not found to my users in case the resource is not present in the DB.

@RestController
public class ExampleController {

    public final ExampleService exampleService;

    @Autowired
    public ExampleController(ExampleService exampleService) {
        this.exampleService = exampleService;
    }

    @ResponseStatus(value = HttpStatus.NOT_FOUND)
    public class ResourceNotFoundException extends RuntimeException {
    }

    @GetMapping("/api/mappings/get-by-id/{id}")
    public ExampleDto getExampleById(@PathVariable String id) {
        Example example = exampleService.findById(id).orElse(null   );
        if (example == null) throw new ResourceNotFoundException();
        return new ExampleDto(example);
    }

}

I would like to know if this code can be considered good and robust enough or if it can be improved and how.


texts:
I'm new to Rust, coming from C++, and so to increase my knowledge of Rust I made a small program to manage an array. You can add, insert, delete, print and more an array or its elements.

I don't really care much about performance at this point, but more on readability, good practices and so on.

So, what could I have done better?

use std::io;

fn get_element() -> i64 {
    println!("Enter the element: ");

    let mut element = String::new();
    io::stdin().read_line(&mut element).expect("Failed to read line!");

    match element.trim().parse() {
        Ok(num) => num,
        Err(_) => {
            println!("Invalid number! Try again.");
            get_element()
        }
    }
}

fn get_index(length: usize) -> usize {
    println!("Enter the index: ");

    let mut index = String::new();
    io::stdin().read_line(&mut index).expect("Failed to read line!");

    match index.trim().parse() {
        Ok(index) if index < length => index,
        Ok(index) => {
            println!("Invalid index! {} is not in range [0, {})! Try again.", index, length);
            get_index(length)
        },
        Err(_) => {
            println!("Invalid number! Try again.");
            get_index(length)
        }
    }
}

fn main() {
    println!("Welcome to the Vector application!");

    let mut vec: Vec<i64> = Vec::new();

    loop {
        println!("=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--=-=-");
        println!("What do you want to do?");
        println!("1. Clear the vector");
        println!("2. Add element");
        println!("3. Insert element at pos");
        println!("4. Delete an element at index");
        println!("5. Print an element");
        println!("6. Print vector");
        println!("*. Exit");

        let mut input = String::new();
        io::stdin().read_line(&mut input).expect("Failed to read line");
        let input: u8 = match input.trim().parse() {
            Ok(num) if num < 7 => num,
            Ok(num) => {
                println!("{} is not in the range [0, 6]!", num);
                continue;
            }
            Err(_) => {
                break;
            }
        };

        let length = vec.len();

        if input == 1 {
            vec.clear();
        }
        else if input == 2 {
            vec.push(get_element());
        }
        else if input == 3 {
            vec.insert(get_index(length), get_element());
        }
        else if input == 4 {
            vec.remove(get_index(length));
        }
        else if input == 5 {
            println!("The element is: {}", match vec.get(get_index(vec.len())) {
                Some(element) => element,
                None => {
                    println!("Invalid index!");
                    continue;
                }
            });
        }
        else if input == 6 {
            for i in &vec {
                println!("{} ", i.to_string());
            }
        }

    }
}


texts: "The Genuine Sieve of Eratosthenes" in C++14

texts:
The following C++ code implements the "Genuine Sieve of Eratosthenes" algorithm as described in Melissa O'Neill's classic paper.  On my MacBook it computes the first 1,000,000 primes in about 11 seconds.

$ ./a.out | head -1000000 | tail -1
15485863
$

Just looking for general code review comments here. At least two parts of sieverator smell really bad to me, but I'm not sure of the proper way to fix them while still preserving the general "STLishness" of this code. For example, I really want to keep using the ranged for-loop in main.

#include <cassert>
#include <functional>
#include <iostream>
#include <memory>
#include <queue>
#include <utility>

template<typename Int>
class iotarator {
    Int value = 0;
    Int step = 1;
public:
    explicit iotarator() = default;
    explicit iotarator(Int v) : value(v) {}
    explicit iotarator(Int v, Int s) : value(v), step(s) {}
    Int operator*() const { return value; }
    iotarator& operator++() { value += step; return *this; }
    iotarator operator++(int) { auto ret = *this; ++*this; return ret; }

    bool operator==(const iotarator& rhs) const {
        return value == rhs.value && step == rhs.step;
    }
    bool operator!=(const iotarator& rhs) const { return !(*this == rhs); }
};

template<class Int>
class sieverator {
    struct erased_iterator {
        virtual Int dereference() = 0;
        virtual void increment() = 0;
    };
    template<class It>
    class derived_iterator : public erased_iterator {
        It it;
    public:
        derived_iterator(It it) : it(std::move(it)) {}
        Int dereference() override { return *it; }
        void increment() override { ++it; }
    };

    Int m_current;
    std::unique_ptr<erased_iterator> m_ptr;

    explicit sieverator() {}  // used by .end()
public:
    template<class It>
    explicit sieverator(It it) :
        m_current(*it),
        m_ptr(std::make_unique<derived_iterator<It>>(std::move(it)))
    {}
    sieverator begin() { return std::move(*this); }
    sieverator end() const { return sieverator{}; }
    bool operator==(const sieverator&) const { return false; }
    bool operator!=(const sieverator&) const { return true; }

    Int operator*() const {
        return m_current;
    }

    sieverator& operator++() {
        cross_off_multiples_of_prime(m_current);
        do {
            m_ptr->increment();
            m_current = m_ptr->dereference();
        } while (is_already_crossed_off(m_current));
        return *this;
    }

    sieverator& operator++(int) = delete;

private:
    struct pair {
        Int next_crossed_off_value;
        Int prime_increment;
        explicit pair(Int a, Int b) : next_crossed_off_value(a), prime_increment(b) {}
        bool operator<(const pair& rhs) const {
            return next_crossed_off_value < rhs.next_crossed_off_value ? true
                 : next_crossed_off_value > rhs.next_crossed_off_value ? false
                 : prime_increment < rhs.prime_increment;
        }
        bool operator>(const pair& rhs) const { return rhs < *this; }
    };
    template<class T> using min_heap = std::priority_queue<T, std::vector<T>, std::greater<>>;
    min_heap<pair> m_pq;

    bool is_already_crossed_off(Int value) {
        if (value != m_pq.top().next_crossed_off_value) {
            return false;
        } else {
            do {
                auto x = m_pq.top();
                m_pq.pop();
                m_pq.emplace(x.next_crossed_off_value + x.prime_increment, x.prime_increment);
            } while (value == m_pq.top().next_crossed_off_value);
            return true;
        }
    }

    void cross_off_multiples_of_prime(Int value) {
        m_pq.emplace(value * value, value);
    }
};

int main()
{
    iotarator<__int128_t> iota(2);
    sieverator<__int128_t> sieve(iota);
    for (int p : sieve) {
        std::cout << p << std::endl;
    }
}


texts:
I'm posting my code for a LeetCode problem copied here. If you would like to review, please do so. Thank you for your time!
Problem

Convert a non-negative integer to its English words representation.
Given input is guaranteed to be less than 2^31 - 1.

Inputs
123
1234567891
151
1414312
1234
1241234113

Outputs
"One Hundred Twenty Three"
"One Billion Two Hundred Thirty Four Million Five Hundred Sixty Seven Thousand Eight Hundred Ninety One"
"One Hundred Fifty One"
"One Million Four Hundred Fourteen Thousand Three Hundred Twelve"
"One Thousand Two Hundred Thirty Four"
"One Billion Two Hundred Forty One Million Two Hundred Thirty Four Thousand One Hundred Thirteen"

Code
#include <string>

class Solution {
    std::string zero_to_twenty[20] = {"Zero", "One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine", "Ten", "Eleven", "Twelve", "Thirteen", "Fourteen", "Fifteen", "Sixteen", "Seventeen", "Eighteen", "Nineteen"};
    std::string tens_and_zero[10] = {"Zero", "Ten", "Twenty", "Thirty", "Forty", "Fifty", "Sixty", "Seventy", "Eighty", "Ninety"};
    std::string SPACE = " ";
    std::string BILLION = "Billion";
    std::string MILLION = "Million";
    std::string THOUSAND = "Thousand";
    std::string HUNDRED = "Hundred";
    std::string ZERO = "Zero";
    std::string EMPTY_STRING = "";
    static inline constexpr int one_billion = 1e9;
    static inline constexpr int one_million = 1e6;
    static inline constexpr int one_thousand = 1e3;
    static inline constexpr int one_hundred = 100;
    static inline constexpr int twenty = 20;
    static inline constexpr int ten = 10;

public:
    inline std::string numberToWords(int num) {
        if (num == 0) {
            return ZERO;

        } else {
            std::string words = int2string(num);
            return words.substr(1, words.length() - 1);
        }
    }

private:
    inline std::string int2string(const int n) {
        if (n >= one_billion) {
            return int2string(n / one_billion) + SPACE + BILLION + int2string(n % one_billion);

        } else if (n >= one_million) {
            return int2string(n / one_million) + SPACE + MILLION + int2string(n % one_million);

        } else if (n >= one_thousand) {
            return int2string(n / one_thousand) + SPACE + THOUSAND + int2string(n % one_thousand);

        } else if (n >= one_hundred) {
            return int2string(n / one_hundred) + SPACE + HUNDRED + int2string(n % one_hundred);

        } else if (n >= twenty) {
            return SPACE + tens_and_zero[n / ten] + int2string(n % ten);

        } else if (n >= 1) {
            return SPACE + zero_to_twenty[n];

        } else {
            return EMPTY_STRING;
        }
    }
};

References

Problem

Solution

Discuss


texts: Best way to retrieve music metadata from audio files?

texts:
I am using the music metadata-browser npm package to retrieve audio metadata from files. This library is being used in an Electron and React desktop app.

To get the metadata of audio files, (and add it to the redux state), I use the following function to get metadata and add the file to redux state. This method works well with small amounts of data, but gets really slow (obviously) as more audio files are given to be processed. Is there a better way I can process these files? Not sure if javascript has any worker/job techniques I could use.

import * as mm from 'music-metadata-browser';

const addFile = (filePath, file, dispatch, ext) => {
  if (Buffer.isBuffer(file)) {
    mm.parseBuffer(file, ext)
      .then(metadata => {
        const libraryEntry = createLibraryEntry(filePath, metadata);

        dispatch({ type: constants.ADD_FILE, libraryEntry, totalFiles });
      });
  }
  else {
    mm.parseBlob(file, ext)
      .then((metadata) => {
        const libraryEntry = createLibraryEntry(filePath, metadata);

        dispatch({ type: constants.ADD_FILE, libraryEntry, totalFiles });
      });
  }
};

const processFiles = (files, dirPath, dispatch) => {
  files.map((file, index) => {
    const parsedFile = file.split('.');
    const format = parsedFile[parsedFile.length - 1];
    const filePath = `${dirPath}/${file}`;
    const isDirectory = fs.lstatSync(filePath).isDirectory();
    if (isDirectory) {
      fs.readdir(filePath, (err, files) => {processFiles(files, filePath, dispatch);});
    }
    else if (isValidFormat(format)) {
      fs.readFile(filePath, (err, file) => {
        addFile(filePath, Buffer(file), dispatch, format);
      });
    }
    return;
  });
};

Thanks for the help.


texts:
I wrote a function for creating all possible translations of a source string, based on a multiple translation map. It works, but generates a lot of intermediate maps (see line marked with *). Is there a better solution? Maybe a more general solution to the "all possible combinations" problem?:

/**
 *
Add to 'targets' all possible variations of 'source', with replacements defined by 'translationsMap'.
 *
For example, after the following call:
 *
addAllReplacements("put the X on the Y", {X => {"cake","egg"}, Y => {"table","chair"}, targets)
 *
targets should contain the following strings:
 *
"put the cake on the table", "put the cake on the chair", "put the egg on the table", "put the egg on the chair"
 */
public static void addAllReplacements (String source, Map<String, Set<String>> translationsMap, Set<String> targets) {
    if (translationsMap.isEmpty()) {
        // no more replacements:
        targets.add(source);
    } else {
        // pick an entry from the map:
        Entry<String, Set<String>> e = translationsMap.entrySet().iterator().next();
        String text = e.getKey();
        Set<String> translations = e.getValue();

        // Create a new translation map, without the currently translated text, for the recursive call:
        HashMap<String, Set<String>> newTranslationMap =
                         new HashMap<String, Set<String>>(translationsMap);  // *
        newTranslationMap.remove(text);

        for (String translation: translations) {
                    addAllReplacements(
                        source.replace(text, translation),
                        newTranslationMap,
                        targets);
        }
    }
}


texts: Convert decimal to binary and find the maximum number of consecutive 1's in the binary

texts:
Task: given a base- 10 integer, n, convert it to binary (base-2). Then find and print the base- 10 integer denoting the maximum number of consecutive 1's in n's binary representation.

What can you say about my implementation of the problem?

n = gets.strip.to_i
array = []
while n > 0
  array.push(n % 2)
  n = n / 2
end

counter = 0
consec = []
len = array.length

array.each_with_index do |x, index|
  if x == 1
    counter += 1
    consec.push(counter) if index + 1 == len
  else
    consec.push(counter) if counter > 0
    counter = 0
  end
end

puts consec.sort[-1]


texts:
Combinations is how many ways to take k elements from n element.  Combination is different from permutation in that order does not matter in combination. Combination is used a lot in poker calculations. Those are combinations used a lot in poker that I know are correct.

In Fast there are a number of terms that cancel out so I avoid them.

Please review for style and efficiency.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Diagnostics;
using System.Numerics;

namespace CodeReview01
{
    class Program
    {
        static void Main(string[] args)
        {
            for (int i = 2; i <= 10; i++)
            {
                BigInteger factorial = Factorial(i);
                Debug.WriteLine($"factorial({i}) =  {factorial}");
            }
            Debug.WriteLine("");
            int n = 3;
            int k = 2;
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFact(n, k)}");
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFast(n, k)}");
            n = 4;
            k = 2;
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFact(n, k)}");
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFast(n, k)}");
            n = 45;
            k = 2;
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFact(n, k)}");
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFast(n, k)}");
            n = 52;
            k = 2;
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFact(n, k)}");
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFast(n, k)}");
            n = 52;
            k = 5;
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFact(n, k)}");
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFast(n, k)}");
            n = 52;
            k = 7;
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFact(n, k)}");
            Debug.WriteLine($"Combin({n}, {k}) = {CombinationFast(n, k)}");
            Debug.WriteLine("");
        }

        public static BigInteger Factorial (int n)
        {
            if(n < 1)
            {
                throw new IndexOutOfRangeException();
            }
            BigInteger factorial = 1;
            for(int i = 2; i <= n; i++)
            {
                factorial *= i;
            }
            return factorial;
        }
        public static int CombinationFact(int n, int k)
        {
            if(n < k)
            {
                throw new IndexOutOfRangeException();
            }
            if (n < 1 | k < 1)
            {
                throw new IndexOutOfRangeException();
            }
            if (n == k)
            {
                return 1;
            }
            int combination = (int)(Factorial(n) / Factorial(k) / Factorial(n - k));
            return combination;
        }
        public static int CombinationFast(int n, int k)
        {
            BigInteger combination = 1;
            //   n * (n-1) * (n-2) * (n-3) * (n - 4) ... 2 * 1/
            // / k * (k-1) ... * 2 * 1
            // / (n-k) * ... 2 * 1_
            for(int i = k+1; i <=n; i++)
            {
                combination *= i;
            }
            BigInteger denominator = 1;
            for (int i = 2; i <= (n - k); i++)
            {
                denominator *= i;
            }
            combination /= denominator;
            return (int)combination;
        }
    }
}


texts: Find an intersection point of 2 line segments

texts:
I want to find an intersection point of two line segments, if one exists. Each line segment is represented by two endpoints. Each endpoint is represented as an ordered pair of numbers. There are no guarantees regarding the line segments (e.g., one or both line segments can be vertical or horizontal).

If multiple points of intersection exist, randomly return one of those intersection points. If no intersection exists, return None.

Pointing out any errors or potential improvements would be really appreciated.

from __future__ import division
from collections import namedtuple

Point = namedtuple('Point', 'x y')
Segment = namedtuple('Segment', 'ep1 ep2')

def find_slope(segment):
    """
    Return the slope of the line segment, if it is defined. If it is
    undefined, return None.
    """
    p1, p2 = segment.ep1, segment.ep2
    if p2.x - p1.x == 0.0:
        return None
    else:
        return (p2.y-p1.y) / (p2.x-p1.x)

def find_y_intercept(slope, point):
    """
    Return the y-intercept of an infinite line with slope equal to slope that
    passes through point. If slope does not exist, return None.
    """
    if slope is None:
        return None
    else:
        return point.y - slope*point.x

def order_segment(segment):
    """
    Order endpoints in segment primarily by x position, and secondarily by y
    position.
    """
    ep1, ep2 = segment.ep1, segment.ep2
    if (ep1.x > ep2.x or ep1.x == ep2.x and ep1.y > ep2.y):
        segment.ep1, segment.ep2 = segment.ep2, segment.ep1

def order_segments(segments):
    """
    Order segments by each segment's first endpoint. Similar to order_segment,
    order primarily by first endpoint's x position, and secondarily by first
    endpoint's y position.
    """
    seg1, seg2 = segments
    if (seg1.ep1.x > seg2.ep1.x or seg1.ep1.x == seg2.ep1.x
            and seg1.ep1.y > seg2.ep1.y):
        segments[0], segments[1] = segments[1], segments[0]

def on(point, segment):
    """
    Return True if point lies on segment. Otherwise, return False.
    """
    return (within(segment.ep1.x, point.x, segment.ep2.x) and
            within(segment.ep1.y, point.y, segment.ep2.y))

def within(p, q, r):
    """
    Return True if q is between p and r. Otherwise, return False.
    """
    return p <= q <= r or r <= q <= p

def find_intersection(segment1, segment2):
    """
    Return an intersection point of segment1 and segment2, if one exists. If
    multiple points of intersection exist, randomly return one of those
    intersection points. If no intersection exists, return None.
    """
    [s1, s2] = [find_slope(l) for l in [segment1, segment2]]
    [k1, k2] = [find_y_intercept(s, l[0])
                for s, l in [(s1, segment1), (s2, segment2)]]

    if s1 == s2:
        if k1 != k2:
            return None
        #  at this point, the two line segments are known to lie on the same
        #  infinite line (i.e. all of the endpoints are collinear)
        segments = [segment1, segment2]
        for segment in segments:
            order_segment(segment)
        order_segments(segments)
        intersection = segments[1].ep1
    else:
        #  assume segment 1 has slope and segment 2 doesn't
        s, x, k = s1, segment2.ep1.x, k1
        #  assumption wrong, segment 1 doesn't have a slope, but segment 2 does
        if s1 is None:
            s, x, k = s2, segment1.ep1.x, k2
        #  assumption wrong, segments 1 and 2 both have slopes
        elif s2 is not None:
            x = (k2-k1) / (s1-s2)
        y = s*x + k
        intersection = Point(x, y)

    if on(intersection, segment1) and on(intersection, segment2):
        return intersection
    else:
        return None


texts:
The problem is：

  Intersection: Given two (singly) linked lists, determine if the two
  lists intersect. Return the intersecting node. Note that the
  intersection is defined based on reference, not value.

  That is, if the kth node of the first linked list is the exact same
  node (by reference) as the jth node of the second linked list, then
  they are intersecting. Below is an example:

The code I wrote is as following, may I know any improvement can be done?

  def intersecton[A](alist: List[A], blist:List[A]): Option[A]= (alist.last, blist.last)match{
    case (a,b) if (a!=b) => None
    case (_,_) if (alist.size>1 && blist.size>1 && alist(alist.size-2) != blist(blist.size-2)) => Some(alist.last)
    case (a,b) if (alist.size ==1||blist.size ==1)&& a==b => Some(a)
    case (_,_) => intersecton(alist.dropRight(1), blist.dropRight(1))
  }

Below is the test case I already run:

val list1: List[Int]=List(5,6, 1, 1, 4)
val list2: List[Int]=List(6, 1, 1,4)
>>6

val list1: List[Int]=List(3,2,4)
val list2: List[Int]=List(5,6,7)
>>None

val list1: List[Int]=List(4,6,7)
 val list2: List[Int]=List(5,6,7)
>>6


texts: Creating a sorted hash counting the occurrences of a number in an array

texts:
I figured something like this:

a= [1, 2, 3, 2, 2, 2, 3, 1, 1, 1, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
a.inject({}) { |h, el| h[el] = a.count(el) if h[el].nil? ; h }.sort_by {|k,v| v}.reverse.inject({}) { |h, el| h[el[0]] = el[1] ; h}

This outputs:

=> {0=>10, 4=>9, 2=>5, 1=>4, 3=>2}

But it seems like a clunky solution that does to many passes over the "same" thing.

What would be a good way to do this from different viewpoints?

I.e:

an "impress your Java friends" (small one-liner not used in production)
Performance
Easily read


texts:
The crawler crawls for a set of keywords and saves the count in a database:

import re
import time
from bs4 import BeautifulSoup
from bs4 import SoupStrainer
import os
import httplib2
#import Links
#import Keywords
import MySQLdb
import peewee
from peewee import *
from datetime import datetime
import argparse
import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
fh = logging.FileHandler('crawler.log')
fh.setLevel(logging.DEBUG)
#ch = logging.StreamHandler()
#ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
#ch.setFormatter(formatter)
#logger.addHandler(ch)
fh.setFormatter(formatter)
logger.addHandler(fh)

parser = argparse.ArgumentParser()

parser.add_argument('-l', '--url', help="The base link to be crawled", required=True)
parser.add_argument('-k', '--keywords', help="Keywords to search", required=True)

args = parser.parse_args()

keywords = (args.keywords).split(',')

mapping  = dict()

mapping[args.url] = keywords

logger.info(mapping)

db = MySQLDatabase('WebSpider', user='ruut', passwd='ruut')

parsed = set()

class DATA(peewee.Model):
    parent_link = peewee.CharField()
    sub_link  = peewee.CharField()
    keyword = peewee.CharField()
    count =  peewee.IntegerField()
    class Meta:
        database = db
        db_table = 'DATA'

def make_soup(s):
   match=re.compile('https://|http://')
   if re.search(match,s):
    try:
         http = httplib2.Http()
             status, response = http.request(s)
             page = BeautifulSoup(response,'lxml')
         return page
    except:
         return None
   else:
     return None

def get_list_of_urls(url):
    match = re.compile('(https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9]\.[^\s]{2,})')
    soup = make_soup(url)
    l = set()
    try:
        for a in soup.find_all('a'):
                try:
                if '?' not in a['href'] and re.search(match,a['href']) and re.search(re.compile(url),a['href']) and a['href']!=url:
                    l.add(str(a['href']))
                except Exception as e:
                logger.info('Exception ' + str(a)+' has no href')
                logger.info(e)
                continue
    except Exception as e:
        logger.info('Exception ' + url+'    has no links')
        logger.info(e)
        pass
    return l

def get_all_the_urls(base,list_of_urls,depth):
    logger.info(depth)
    if depth == 10:
        return
    else:
        depth = depth  + 1
        for i in list_of_urls: #scan the list of urls
            s = get_list_of_urls(i)
            get_all_the_urls(base,s,depth)
            for j in s: #scan the sublinks
               try:
                    if j in parsed:
                   continue
                soup = make_soup(j)
                logger.info('url is     '+ j)
                for k in mapping[base]:  #look for keys on the webpage
                    key_count = len(soup(text=re.compile(k, re.IGNORECASE)))
                    logger.info('Key count is '+str(key_count))
                    if(key_count>0):
                        record = DATA(parent_link = base,sub_link = j ,keyword = k ,count = key_count) #i,j,k,key_count
                            record.save()
                        parsed.add(j)
                        logger.info('saved data successfully ' +str(key_count))
               except Exception as e:
                logger.info('Exception ' +str(e)+'  in keywords searching')
                continue

def populate_db():
    k = set()
    k.add(args.url)
    temp = time.time()
    logger.info(str(datetime.now()))
    get_all_the_urls(args.url,k,0)
    logger.info('time taken '+str(time.time()-temp))
populate_db()


texts:
I've created a crawler which is scraping name, phone number and web address of each profile from houzz website. Hope I did it the right way. Here is what I've written:

import requests
from lxml import html

url="https://www.houzz.com/professionals/"

def scraper_func(mainurl):
    response = requests.get(mainurl).text
    tree = html.fromstring(response)
    for titles in tree.xpath("//a[@class='sidebar-item-label']"):
        link = titles.xpath(".//@href")
        for item in link:
            paging_stuff(item)

# Done crawling links to the category from left-sided bar
def paging_stuff(process_links):
    response = requests.get(process_links).text
    tree = html.fromstring(response)
    for titles in tree.xpath("//ul[@class='pagination']"):
        link = titles.xpath(".//a[@class='pageNumber']/@href")
        for item in link:
            processing_stuff(item)

# Going to each page to crawl the whole links spread through pagination connected to the profile page
def processing_stuff(procured_links):
    response = requests.get(procured_links).text
    tree = html.fromstring(response)
    for titles in tree.xpath("//div[@class='name-info']"):
        links = titles.xpath(".//a[@class='pro-title']/@href")[0]
        main_stuff(links)

# Going to the profile page of each link
def main_stuff(main_links):
    response = requests.get(main_links).text
    tree = html.fromstring(response)

    def if_exist(titles,xpath):
        info=titles.xpath(xpath)
        if info:
            return info[0]
        return ""

    for titles in tree.xpath("//div[@class='profile-cover']"):
        name = if_exist(titles,".//a[@class='profile-full-name']/text()")
        phone = if_exist(titles,".//a[contains(concat(' ', @class, ' '), ' click-to-call-link ')]/@phone")
        web = if_exist(titles,".//a[@class='proWebsiteLink']/@href")
        print(name,phone,web)

scraper_func(url)


texts:
I'm looking for some feedback on my first 'real' Python program. Started with Python a few days ago and wanted to get right into it and make something hands on.
Currently the program lets you download audio and video for a YouTube video using Pytube. It allows you specify the location to save the download, whether to download Audio or Video and what resolution to download videos at.

#global variables
yt = None
res = None
stream = None
file_path = None
av_swtich = None

#see if the user wants to download audio or video

def av_select():
    global av_switch
    print('Would you like to download Video or Audio:')
    while True:
        av_switch = input('Please enter \'1\' for Video and \'2\' for Audio\n')
        if av_switch not in ('1', '2'):
            print('Pleae make a valid selection')
            continue
        else:
            break

#Download Audio Stream

def audio_download():
    global yt
    global file_path
    print('The Video has the following audio streams')
    print(*yt.streams.filter(only_audio=True), sep='\n')
    while True:
        itag = input('Please Enter the Itag of the stream you want to download:\n')
        try:
            yt.streams.get_by_itag(str(itag)).download(output_path=str(file_path), filename_prefix='Audio-')
            print('The audio stream has been downloaded')
            return True
        except:
            print('You have entered an invalid Itag')
            continue

#Define Download location
def download_location():
    global file_path
    while True:
        file_path = input('Please enter the Folder to save the Video in:\n')
        x = os.path.exists(path = str(file_path))
        if x == True:
            break
        else:
            print('The specified Folder does not exit')
            continue

#Select the video that the user wants to use
def video_select():
    global yt
    while True:
        video = input('Please enter the video URL:\n')
        try:
            yt = YouTube(video)
            print('The title of the video you selected is: ' + yt.title)
            return False
        except:
            print('The URL is not Valid\n')
            continue

#Select resoultion and display possible stings
def resoultion_select():
    global res
    global res_list
    while True:
        res = input('Please enter a Resoultion:\n')
        try:
            res_list = yt.streams.filter(resolution= str(res))
            return False
        except:
            print('You have entered an invalid resoultion')
            continue

#Select the stream to download based on the itag
def download_stream():
    global file_path
    global yt
    global res_list
    print('The Video has the following streams')
    print(*res_list, sep='\n')
    while True:
        itag = input('Please Enter the Itag of the stream you want to download:\n')
        try:
            yt.streams.get_by_itag(str(itag)).download(str(file_path))
            print('The video has been downloaded')
            return True
        except:
            print('You have entered an invalid Itag')
            continue

def Main():
    global av_switch
    print('Welcome to the Youtube Video Downloader\n')
    download_location()
    video_select()
    av_select()
    if av_switch == '1':
        resoultion_select()
        download_stream()
    else:
        audio_download()

Main()


texts: Project Euler #9 in C: Special Pythagorean triplet

texts:
I have finished Project Euler #9 on HackerRank:

  Given \$N\$, Check if there exists any Pythagorean triplet for which \$a+b+c=N\$, \$a^2+b^2=c^2\$.
  Find maximum possible value of \$abc\$ among all such Pythagorean triplets, and if there is no such Pythagorean triplet print −1.

However, I feel uncomfortable about my code. I think it is sort of spaghetti code that is neither well organized nor fast (just a feeling).

Optimization and simplification is needed. (For example, there are many nested ifs).

#include <stdio.h>
#include <time.h>
int main()
{
  clock_t start = clock();
  unsigned int i,j,t,n;
  long p;
  scanf("%u",&t);
  while(t>0)
  {
    p =-1;
    i=2;
    j=1;
    scanf("%u",&n);
    while(1)
    {
      int s = 2*i*i+2*i*j;
      if(s ==n )
      {
        long o = 2*i*j*(i*i+j*j)*(i*i-j*j);
        if (p<o) p =o;
        if(i>j+1) j++;
        else
        {
          j=1;
          i++;
        }
      }
      else if(s>n)
      {
        if(j==1) goto end;
        j=1;
        i++;
      }
      else
      {
        int counter;
        for(counter =2; s*counter<=n;counter++)
        {
          if(s*counter == n)
          {
            long o = 2*i*j*(i*i+j*j)*(i*i-j*j)*counter*counter*counter;
            if (p<o) p =o;
          }
        }
        if(i>j+1) j++;
        else
        {
          j=1;
          i++;
        }
      }
    }
    end: printf("%li\n",p);
    t--;
  }
  clock_t end = clock();
  printf("time in milliseconds %lf\n",(double)(end-start)*1000/CLOCKS_PER_SEC);
  return 0;
}


texts:
I am implementing a binary heap in Java. I've come to the point where I have to fix the heap ordering whenever a new node is being inserted to the heap. The code below fixes the heap ordering. I have a gut feeling that the code can be improved.

private void fixHeapOrder(Node anomalousNode)
{
    if (this.heapType == 0) {
        // Min-heaps
        while (anomalousNode.getParent().getValue() > anomalousNode.getValue()) {
            // In min-heaps, the parent's value must be less than or equal to the
            // value of its children.

            Node.swapValues(anomalousNode.getParent(), anomalousNode);
            anomalousNode = anomalousNode.getParent();
        }
    } else if (this.heapType == 1) {
        // Max-heaps
        while (anomalousNode.getParent().getValue() < anomalousNode.getValue()) {
            // In max-heaps, the parent's value must be greater than or equal to the
            // value of its children.

            Node.swapValues(anomalousNode.getParent(), anomalousNode);
            anomalousNode = anomalousNode.getParent();
        }
    }
}

How else can I improve the code?


texts:
I wrote a program that is supposed to be a breadth-first search algorithm, but I'm very new to search algorithm so I don't know if my method is very effective or if there is a simpler way to do this. So I'm asking if there is a way to improve my code.

In this case, I have made the program search for the closest 1 in a 2d list where you can enter the start position and it will then find the shortest path between the start position and the 1. Of course, this can be changed in a variety of ways.

I tried to make the variables and function names as clear as possible, so I don't think I need to explain everything about my code, but that's easy for me to say so please comment if you need clarification.

m = [[1, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 1, 0],
     [0, 0, 1, 0, 0, 0],
     [0, 0, 0, 0, 0, 1],
     [0, 0, 0, 0, 1, 0],
     [1, 0, 1, 0, 0, 0],
     [1, 0, 0, 0, 0, 1],
     [0, 0, 0, 0, 1, 0],
     [1, 0, 1, 0, 0, 0]]

visited = []
queue = []

parent = {}
num = 0

def breadth_first(maze, x, y):
    queue.append((x, y))
    visited.append((x, y))

    while queue:
        pos = queue[0]
        x = pos[0]
        y = pos[1]

        # remove from queue
        queue.remove(pos)

        # find neighbor
        for dir_x, dir_y in ((-1, 0), (1, 0), (0, -1), (0, 1)):
            newx = x + dir_x
            newy = y + dir_y

            neighbor = (newx, newy)

            # add to queue
            if len(maze[0]) > newx >= 0 and len(maze) > newy >= 0 and neighbor not in visited and neighbor not in queue:
                if m[newy][newx] == 1:
                    parent[neighbor] = pos
                    return neighbor

                queue.append(neighbor)
                visited.append(neighbor)
                parent[neighbor] = pos

closest = breadth_first(m, 0, 0)

path = [closest]

def search(traceback):
    while traceback != (0, 0):
        for key, value in parent.items():
            if traceback == key:
                path.append(value)
                traceback = value

    return path

def solved(maze, input_path):
    for pos in input_path[1:-1]:
        maze[pos[1]][pos[0]] = '+'

    return maze

print(solved(m, search(closest)))


texts: How to efficiently loop through paragraphs and make simple changes with Word VBA (Specially reverse loop to Delete Paragraphs)

texts:
This is regarding my answer with SO post How to remove paragraph marks with different format in MS-Word.
My primary question, is there any way performance of the code could be improved to operate on document as intended by OP (2.2 MB and has 2.1K pages, 871K words, 4,6M characters including spaces)?
On Secondary, is there a simple way or workaround that I am missing (and making things unnecessary complicated) to perform the sane task efficiently?

Here I reproduce my code after adding some futile measures to improve performance of the code with file of the size specified by the OP.

    Option Explicit
    Sub ReplacePara()
    Dim Para As Paragraph, Xstr As String, Rng As Range
    Dim i As Long, ln As Long, tm As Double, PrCnt As Long
    Dim PrvChrSize As Integer, NextChrSize  As Integer
    Dim PrvChrFont As String, NextChrFont  As String
    Dim PrvChrItalic As Boolean, NextChrItalic As Boolean
    tm = Timer

‘Following measures added to improve performance
‘but on the contrary it’s found instead of increasing time taken
    Application.ScreenUpdating = False
        With Options
        .Pagination = False
        .CheckSpellingAsYouType = False
        .CheckGrammarAsYouType = False
        End With

        With ActiveDocument
        PrCnt = .Paragraphs.Count
        Debug.Print PrCnt
        For i = .Paragraphs.Count To 1 Step -1
        Set Para = .Paragraphs(i)
        ln = Para.Range.Characters.Count

            If ln > 1 Then
                With Para.Range.Characters(ln - 1).Font
                PrvChrSize = .Size
                PrvChrFont = .Name
                PrvChrItalic = .Italic
                End With

                If i < .Paragraphs.Count Then
                    With .Paragraphs(i + 1).Range.Characters(1).Font
                    NextChrSize = .Size
                    NextChrFont = .Name
                    NextChrItalic = .Italic
                    End With
                Else
                NextChrSize = 0
                NextChrFont = ""
                NextChrItalic = False
                End If
            End If

            'Debug.Print i, PrvChrSize, PrvChrFont, NextChrSize, NextChrFont
            If (PrvChrSize = 15 And (PrvChrFont = "Arial" Or PrvChrItalic = True)) _
            And (NextChrSize = 15 And (NextChrFont = "Arial" Or NextChrItalic)) Then
            Para.Range.Characters(ln).Text = " "
            End If
        .UndoClear
        'If PrCnt < 1000 Then Debug.Print i & "/" & PrCnt
        Next
      End With

        With Options
        .Pagination = True
        .CheckSpellingAsYouType = True
        .CheckGrammarAsYouType = True
        End With
    Application.ScreenUpdating = True
    Debug.Print " Seconds taken:" & Timer - tm
    End Sub

The added measures actually found to increase time taken (from 3 odd minutes to 4 odd minutes) with documents of 124 pages. I haven’t ventured far to go for LockWindowUpdate API.

Though the code tested Ok with documents of 100 pages around. I could not finish the task with a makeshift giant file of around 2.4 K pages. It is virtually crashing Word (not recovering from ‘Not responding mode’). I ceated the file with a simple code stub with the sample file linked by OP in the SO post. Code stub was also produced for ease is testing.

Sub makebig()
Dim Rng As Range, MyRange As Range
Dim Wd As Document
Dim i  As Long
Set Wd = ThisDocument
Set Rng = Wd.Content
Rng.Copy

    For x = 1 To 2000
    Set MyRange = Wd.Content
    MyRange.EndOf Unit:=wdStory, Extend:=wdMove
    MyRange.Paste
    Next
End Sub

Running the code with sample file twice (1st time with For x = 1 To 2000 and second time with For x = 1 To 1) will produce a file about 2.4 K pages. For getting a file of 124 pages from the sample file 200 loops are sufficient.


texts:
This code works, but I'm curious what feedback you have about style and doing things in a more "pythonic" way.

def printUsage():
    print("Usage: goodproxy.py -f <proxyfile> -u <url> -t <timeout>")

def main(argv):

    filename    = ""
    url         = ""
    timeout     = None

    try:

        # USE GETOPS FOR EASIER PARAMETER PROCESSING
        opts, args = getopt.getopt(argv, "f:u:t:",["file=", "url=", "timeout="])

    except getopt.GetoptError:

        printUsage()
        sys.exit(2)

    # CHECK FOR MISSING PARAMETERS
    # THE 'OPTS' OBJECT CONSISTS OF A LIST OF TUPLES LIKE (PARAM,VAL)
    # LIST COMPREHENSION COLLECTS FIRST ELEMENTS FROM EACH TUPLE INTO A NEW LIST
    # THEN TESTS THE NEW LIST FOR THE REQUIRED PARAMS

    if not any(f in [opt[0] for opt in opts] for f in ['-f','--file']):
        printUsage()
        print("Error: -f parameter missing")
        sys.exit(2)

    if not any(u in [opt[0] for opt in opts] for u in ['-u','--url']):
        printUsage()
        print("Error: -u parameter missing")
        sys.exit(2)

    if not any(t in [opt[0] for opt in opts] for t in ['-t','--timeout']):
        printUsage()
        print("Error: -t parameter missing")
        sys.exit(2)

    # CONFIGURE SETTINGS BASED ON THE PASSED IN PARAMETERS
    for opt, arg in opts:

        if opt in ('-f', '--file'):

            print("Using proxies in: " + arg)
            filename = arg

        elif opt in ('-u', '--url'):

            print("Using URL: " + arg)
            url = arg

        elif opt in ('-t', '--timeout'):

            print("Timeout: {0}s".format(arg))
            timeout = arg

if __name__ == "__main__":
    main(sys.argv[1:])

I also found PyLint to assist with improving styling and code smells.


texts:
I'm trying to learn more efficient coding so I'm doing some challenges. I have a challenge where I'm given some hex data. I know that hex data is XOR encoded with a four character long ASCII key. The only thing I know about the clear text data is that it contains "a lot of '#' and space characters".

I don't know what they mean by a lot and there are many possibilities that end up with more than 50 hash tags and spaces, though most of them have less than 10 of each. So I figured the only way I could do this would be to narrow down the number of possibilities.

In the following code I'm iterating through every possible combination of 4 character ASCII using the Cartesian product (keys). I'm using a namedtuple to store the number of hashtags and spaces for each key that has more than 10 of either. I'm hoping this would help me determine good potential candidates to manually sift through.

import itertools,re,string
from collections import namedtuple

# key is four ascii chars long so we will iterate through data 4-char chunk at a time
data = re.findall('....?', '\x6e\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x47\x45\x50\x02\x44\x46\x53\x01\x44\x45\x50\x02\x47\x45\x50\x01\x44\x46\x53\x02\x47\x45\x50\x02\x6e\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x47\x45\x53\x01\x47\x45\x53\x01\x44\x45\x50\x01\x44\x46\x53\x01\x44\x46\x50\x02\x44\x46\x53\x01\x6e\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x47\x45\x53\x01\x44\x45\x53\x01\x44\x45\x50\x02\x47\x45\x50\x01\x44\x46\x53\x02\x47\x45\x50\x01\x6e\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x47\x45\x53\x01\x47\x45\x53\x01\x44\x45\x50\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x50\x02\x6e\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x47\x45\x50\x02\x44\x46\x53\x01\x44\x45\x50\x01\x44\x46\x53\x01\x44\x46\x50\x02\x47\x45\x50\x01\x6e\x6c\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x6c\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x45\x50\x01\x44\x46\x30\x4e\x09\x0b\x06\x4f\x0d\x05\x12\x55\x0d\x09\x1d\x52\x44\x25\x1c\x4f\x10\x14\x1c\x4d\x44\x46\x53\x02\x47\x6c\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x46\x53\x01\x44\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x45\x50\x02\x47\x6c\x79\x6c\x01\x08\x06\x2b\x49\x4b\x5e\x0c\x6e\x6c\x42\x0f\x44\x35\x07\x40\x16\x12\x06\x51\x44\x05\x1c\x4c\x09\x13\x1d\x48\x07\x07\x07\x48\x0b\x08\x00\x0f\x6e\x54\x5d\x01\x37\x0e\x06\x55\x00\x09\x04\x4f\x44\x05\x1c\x4c\x09\x13\x1d\x48\x07\x07\x07\x48\x0b\x08\x00\x0f\x6e\x55\x5d\x01\x28\x09\x14\x4e\x11\x12\x5d\x2b\x6e\x36\x1f\x44\x05\x15\x16\x01\x17\x03\x1f\x44\x07\x12\x53\x58\x0b\x13\x01\x01\x05\x05\x07\x48\x0b\x08\x53\x1f\x6e')

# Cartesian product, all possible printable ascii chars
keys = itertools.product(string.printable, repeat = 4)

# Namedtuple and list used to store good candidates
Possible = namedtuple('Possible', ['key', 'hash', 'space'])
possibles = []

for k in keys:
    clear = ''
    key = ''.join(x for x in k) # product() returns tuples, need to join them
    for word in data:
        clear +=  bytearray(a^b for a, b in zip(*map(bytearray, [word, key])))

    # Count hash tags and spaces and store good candidates
    hcount = clear.count('#')
    scount = clear.count(' ')
    if hcount > 10 or scount > 10:
        possibles.append( Possible(key, hcount, scount) )

# display candidates
for possible in possibles:
    print possible

My code seems to have a very poor performance and I would love to know how I can improve it.


texts: Optimizing unboxed array operations in Haskell

texts:
Consider this simplified code which successively permutes array elements:

import Data.Word
import Data.Bits
import Data.Array.Unboxed
import Data.Array.Base
import Data.Array.ST

test3 :: UArray Int Word32 -> UArray Int Word32
test3 arr = arr `seq` runSTUArray (change arr) where
    change a' = do
        a <- thaw a'
        x0 <- unsafeRead a 0
        x1 <- unsafeRead a 1
        unsafeWrite a 0 (x1 + 1)
        unsafeWrite a 1 (x0 - 1)
        return a

test4 :: (Word32, Word32) -> (Word32, Word32)
test4 (x, y) = x `seq` y `seq` (y + 1, x - 1)

apply :: Int -> (Int -> a -> a) -> a -> a
apply n f v0 = n `seq` v0 `seq` applyLoop 0 v0 where
    applyLoop i v
        | i == n - 1 = res
        | otherwise = applyLoop (i + 1) res
        where res = f i v

main :: IO ()
main = let arr = listArray (0, 1) [0, 1] :: UArray Int Word32
    in print $ apply 10000000 (\_ x -> x `seq` test3 x) arr

On my machine, with ghc 7.4.2 and -O3 flag the version with test4 (plain tuples) shows ~10 times better performance then the version with UArray (also, according to the profiler, UArray version has total allocation of ~1.5 Gb). Are there any other optimizations I could do? I'd prefer to use the array version, because it will be more convenient in the real application.


texts: Automatic creation of python conda packages

texts:
I packaged a pure python library using conda.
It is tested to work with 2.7, 3.5 and 3.6 and should work on all three mayor operating systems.

I am quite surprised, that there is no command or option like conda-build-all. Instead I have to build and convert manually.

So there are two questions:

Is the idea to just put the tasks into a shell script sane? It feels wrong.

Is the shell script written in a maintaineable manner?

#!/bin/sh

# ====================================================
# Please note the bug, that for conda-build the option '--output' does
# not respect the directories given by '--output-folder':
# https://github.com/conda/conda-build/issues/1957
# ====================================================

tmp=$(dirname $(conda-build --output .))
system=$(basename $tmp)
root_dir=$(dirname $tmp)

for py_version in '2.7' '3.5' '3.6'
do
  package_name=$(basename $(conda-build --python ${py_version} --output .))
  package_path="${root_dir}/${py_version}/${system}/${package_name}"

  conda-build --no-anaconda-upload \
              --python ${py_version} \
              --output-folder "${root_dir}/${py_version}" .
  for platform in 'osx-64' 'linux-32' 'linux-64' 'win-32' 'win-64'
  do
    conda-convert -p ${platform} -o "${root_dir}/${py_version}" ${package_path}
    anaconda upload "${root_dir}/${py_version}/${platform}/${package_name}"
  done
done

PS: Here you can find the module and the shell script.


texts:
To become more familiar with templates I attempted a doubly linked list implementation in C++. Is this a valid use for a template?

template <class T>
struct LinkedListNode {
    LinkedListNode<T>* m_previous;
    T m_data;
    LinkedListNode<T>* m_next;
};

template <class T>
class LinkedList {
public:
    LinkedList() {
        m_head = nullptr;
    }

    LinkedListNode<T>* m_head;

    void InsertEnd(T data);
    void InsertFront(T data);

};

template<class T>
inline void LinkedList<T>::InsertFront(T data)
{
    LinkedListNode<T>* insertedNode = new LinkedListNode<T>();
    LinkedListNode<T>* firstNode = m_head;

    if (!firstNode) {
        insertedNode->m_data = data;
        insertedNode->m_next = nullptr;
        insertedNode->m_previous = nullptr;
        m_head = insertedNode;
    }
    else {
        while (firstNode->m_previous != nullptr) {
            firstNode = firstNode->m_previous;
        }

        insertedNode->m_next = firstNode;
        insertedNode->m_previous = nullptr;
        insertedNode->m_data = data;

        firstNode->m_previous = insertedNode;

        m_head = insertedNode;
    }

}

template<class T>
inline void LinkedList<T>::InsertEnd(T data)
{
    LinkedListNode<T>* insertedNode = new LinkedListNode<T>();
    LinkedListNode<T>* lastNode = m_head;

    if (!lastNode) {
        insertedNode->m_data = data;
        insertedNode->m_next = nullptr;
        insertedNode->m_previous = nullptr;
        m_head = insertedNode;

    }
    else {
        while (lastNode->m_next != nullptr) {
            lastNode = lastNode->m_next;
        }

        lastNode->m_next = insertedNode;

        insertedNode->m_previous = lastNode;
        insertedNode->m_next = nullptr;
        insertedNode->m_data = data;
        m_head = insertedNode;
    }
}

main.cpp

LinkedList<int>* linkedList = new LinkedList<int>();
linkedList->InsertEnd(2);
linkedList->InsertEnd(5);
linkedList->InsertEnd(9);
linkedList->InsertFront(1);
linkedList->InsertEnd(10);

LinkedList<std::string>* stringList = new LinkedList<std::string>();
stringList->InsertFront("one");
stringList->InsertEnd("two");
stringList->InsertFront("three");

Ordering should be
linkedList
1,2,5,9,10
stringList
three,one,two


texts:
Is this going to leak memory or is it ok? Please also let me know if there is something else that I need to pay attention to.

typedef struct
{
int len;
UC * message;
}pack;

pack * prepare_packet_to_send(const int length,const unsigned char tag,const int numargs, ... )
{

pack *layer=  malloc(sizeof(pack));
va_list listp;
va_start( listp, numargs );
int step = 0;
layer->message = (unsigned char *) malloc(length);
layer->len = length;
int i = 0;
int len = 0;
unsigned char *source_message ;

for( i = 0 ; i < numargs; i++ )
{

source_message = va_arg( listp, unsigned char *);
len = va_arg( listp, long);
memcpy(layer->message+step, source_message, (long) len);
step+=len;
}

va_end( listp );
 return layer;
}

main()

{

pack *test = call prepare_packet_to_send(sizeof(var1)+sizeof(var2),any tag,any args)

// are following two frees correct/enough? or is there something else i need to do to     prevent mem leak?

free(test->message);
free(test);

}


texts: Accessing a list of dictionaries in a list of dictionaries

texts:
I have 4-deep JSON data structure. Basically its a list of dictionaries nested in a list of dictionaries. This is my first time working with this kind of nested data. Below I have done my best to reproduce the structure:

   data = [
    0 : {"name": "hotel1",
         "overall_rating": 4,
         "reviews": [{"rating": 5, "review" : "best place ever"},
                     {"rating": 1, "review" : "toilet was blocked"}],
        }

    1 : {"name": "hotel2",
         "overall_rating": 4,
         "reviews": [{"rating":4, "review" : "welly nice"},
                      {"rating":3, "review" : "I couldn't bring my dog"}],
        }
    ]

What i want to do is write a function that can iterate over all the nested dictionaries and retrieve "rating" and "review". I have written the following program:

reviews = []
ratings = []

for index in range(len(data)):
    for key in data[index]:
        for key in range(len(data[index]['reviews'])):
            reviews.append(data[index]['reviews'][key]['review'])
            ratings.append(data[index]['reviews'][key]['rating'])

However, this loop only retrieves the ratings and reviews of the first nested.
Additionally, it does this len(data) x len(dict) x len(reviews) times. Besides getting help with the loop I it would also be nice with some pointers to libraries that make dealing with this kind of data structure easier (if such a library exists).


texts:
I'm attempting to create a comparison table to compare products depending on the add-ons they have. To grab the data and filter it I'm using LINQ To SQL.

Table's Layout (cut short):

Products Table
ID
Name
CategoryID
ProductAddons

Category Table
ID
Name

ProductAddon Table
ID
Amount
AddonID
ProductID

Addon Table
ID
Name

Example Data:

Products

ID      Name      CategoryID
1       Product1  1
2       Product2  1
3       Product3  1

Categories

ID      Name
1       Category1

ProductAddons

ID      Amount      AddonID      ProductID
1       1           1            1
2       2           2            1
3       1           3            1
4       2           1            2
5       3           2            2
6       1           2            3
7       1           3            3

Addons

ID      Name
1       Addon1
2       Addon2
3       Addon3

Currently I have this:

var addons = (from s in Products
              where s.Category.Name == "Category1"
              orderby s.ProductAddons.Count descending
              let adds = from a in s.ProductAddons
                         orderby a.Addon.Name
                         select new { Name = a.Addon.Name, Amount = a.Amount }
              select adds).ToList();

var compare = from c in addons
              let has = from z in addons.First().Union(addons.First().Except(c))
                      let add = (from a in c
                                 where a.Name == z.Name
                                 select a.Amount).FirstOrDefault()
                      select new { Name = z.Name, Amount = add }
              select has;

var compileResults = from c in addons.First()
                     let adds = from s in compare
                             let y = (from a in s
                                      where a.Name.Contains(c.Name)
                                      select a.Amount).First()
                             select y
                     select new { Addon = c.Name, Amounts = adds };

This would return (3 products, 3 Add-ons):

Addon          Amounts

Addon1         1
               2
               0

Addon2         2
               3
               1

Addon3         1
               0
               1

I can then loop through the results to build my compare table, which works fine. But to me the LINQ looks somewhat messy. Would there be a cleaner way to do this?


texts: Prevent multiple async calls from all attempting to refresh an expired OAuth token

texts:
I have some code that needs to access an API that requires OAuth authorization in the form of a token, and every time the token expires, it needs to be refreshed.

I made a function called _fetch() that wraps the API calls that I'm making, and will both authorize the requests, and automatically refresh the token if it has expired.

The caveat is that if I call some function that rapidly calls _fetch() \$N\$ amount of times when the token has expired, then reAuth(), which refreshes the token, will also be called \$N\$ amount of times, which is undesirable due to the fact that it'll make \$N\$ API calls to reauthorize the token when only 1 is required.

Reduced to a really small example, possibly with mistakes, the code that I have looks a little like this

import fetch from 'node-fetch'

class Example {
  async reAuth() {
    // refresh the token
    return fetch('authAPI')
  }

  async doMultipleAsyncThings() {
    return await Promise.all([
      _fetch('url1'),
      _fetch('url2'),
      _fetch('url3')
    ])
  }

  async _fetch(url, userOptions) {
    if (tokenIsExpired(this.token)) {
      this.token = await this.reAuth();
    }

    const options = {
      headers: {
        Authorization: `Bearer ${this.token}`
      },
      ...userOptions
    }
    return fetch(url, options);
  }
}

const e = new Example()
e.doMultipleAsyncThings() // reAuth called 3 times if token expired

I think I have a solution to the problem, but I'm not sure if it's overkill or not.

Basically, I would memoize the reAuth() function such that as long as the promise it returns the first time isn't resolved it would return that same promise for all future calls to prevent calling the OAuth API again and again. Then when the promise resolves it would delete itself from the internal cache so that the next time reAuth() is called it would once again actually call the API.

I implemented this here: https://github.com/AriaFallah/weak-memoize/blob/master/src/index.js

// @flow

function memoize(callback: Function): Function {
  const cache = {}
  function memoized(...parameters: Array<mixed>) {
    const cacheKey = JSON.stringify(parameters)

    if (cache[cacheKey]) {
      return cache[cacheKey]
    }

    // Get and add the value to the cache
    const value = callback.apply(this, parameters)
    cache[cacheKey] = value

    if (!value || value.constructor.name !== 'Promise') {
      throw new Error('Memoization Error, Async function returned non-promise value')
    }

    // Delete the value regardless of whether it resolves or rejects
    return value.then(function(internalValue) {
      cache[cacheKey] = false
      return internalValue
    }, function(err) {
      cache[cacheKey] = false
      throw err
    })
  }

  memoized.cache = cache
  return memoized
}

export default memoize

and then to solve the problem all I would need to do is decorate reAuth()

import memoize from 'weak-memoize/promise'
Example.reAuth = memoize(Example.reAuth)

I think the above makes sense, but I'm curious if there is a simpler way to fix the problem I'm facing.


texts: Object oriented design for access control

texts:
Problem Statement:

  Design an employee swap in swap out system.  The system will have a machine which records the swap in and swap out.  The user can also login in a portal where he can check his swap in /swap out time. He can correct his time also.

  There will be managers for employee who can check the entry for all the employees which are under them and correct their subordinates timings also.

  Similarly, a HR person can only view every Employee's records.

I chalked out the following skeleton. I am new to design and would like any pointers on how to approach such problems.

class User{
    String name;
    String designation;
    int id;
}

class AccessGroup{
    int priority;
    String name;
    Boolean read;
    Boolean write;
    //Setter - Getter methods
}

static class AccessManagement{
    HashSet<String, PriorityQueue<AccessGroup> accessControlTable;

    public void addUser(User u, AccessGroup g);
    public void addUserToGroup(User u, AccessGroup g);
    public void removeUser(User u);
    public void removeUserFromGroup(User u, AccessGroup g);
    public AccessGroup getPriorityAccessGroupForUser(User u);
    public PriorityQueue<AccessGroup> getUserGroups(User u);
    public boolean isValidUser(User u);
    public boolean hasReadPermission(User u);
    public boolean hasWritePermission(User u);
}

class Mediator{
    String url="";

    public List<Records> getRecords(User u);
    public void updateRecordForUser(User u, Record r, Value v){
        if(!AccessManagement.isValidUser(u))
            throw Exception();

        if(AccessManagement.hasWritePermission(u)){
            try{
                Connection conn = DriverManager.getConnection(url);
                Statement statement = conn.createStatement();
                statement.executeQuery("Update r in TimeStamps where id= u.id");
            }
            catch(Exception e){
                //Do something
            }
        }
    }
}


texts: Fetch Keywords from comment and assign keyword id to comment

texts:
I have a small callback function which receives a comment in array like below:

                        $data[$count] = array(
                            'comment_id' => $comment['id'],
                            'comment_body' => $comment['message'],
                            'comment_author_id' => $comment['from']['id'],
                            'comment_author_name' => $comment['from']['name'],
                            'active' => 1,
                            'keyword_id' => null,
                            'live_video_id' => $liveVideo->live_vidoe_id
                        );

the I send this comment to a function AssignKeywords as below:

$data = array_map([$this , 'assignKeywords'], $data);
Below is the definition of AssignKeywords

public function assignKeywords($comment)
{
    # code...
    $keywords = Keyword::where('live_video_id' , $comment['live_video_id'])->get();
    $commentBody = strtolower($comment['comment_body']);
    preg_match_all('/(?<!\w)#\w+/',$commentBody,$matches); //it returns an associative array with just 0 index and array of matches on this 0 index

    foreach($matches as $match) {   //only 1
        foreach ($match as $item) {    //max 10 if a user is mad :\
            foreach ($keywords as $keyword) {   //Max 4
                $keywordBody = strtolower($keyword['keyword_name']);
                if ($item === $keywordBody) {
                    if ($comment['keyword_id'] == NULL) { //to check if it already contains a value, if it contains we don't want to assign again
                        $comment['keyword_id'] = $keyword['id'];
                    }
                }
            }
        }
    }

    return $comment;
}<br>

I am looking for a better way to Assign Keywords and minimize foreach loops and maybe solve it by using functions? Any thoughts?


texts:
I've solved the meaty bits of this bioinformatics problem, albeit a bit clumsily I think. In particular I've messed around with manipulating the fasta info into a form Biopython's SeqIO.parse() will accept and it screams to be optimized.

So there is my question to you. How can I better save the HTTP response from Uniprot into something that is easily parsed by SeqIO.parse()?

'''
Stephen Wist
to solve this problem: http://rosalind.info/problems/mprt/
'''

import sys, re # IO and motif finding
import urllib3 # get protein fasta from uniprot
from Bio import SeqIO, Seq # fasta parsing and manipulation

if len(sys.argv)<2:
    print("need at least one uniprotID")

f = open("ros.fa", "w")
http = urllib3.PoolManager() # urllib3 takes care of stuff here

# get fasta of all given uniprotIDs
for item in sys.argv[1:]:
    url = "http://www.uniprot.org/uniprot/" + item + ".fasta"
    req = http.request("GET", url)
    byte_string = req.data
    string = byte_string.decode("utf-8")
    f.write(string) # BioPython SeqIO.parse() won't work on the
                    # decoded byte_string

print(  "\nreq.data:\n", req.data,
        "\ndecoded req.data:\n", string)

f.close()
pattern = re.compile("[N][^P][S|T][^P]")
for seq in SeqIO.parse("ros.fa", "fasta"):
# keeping this here, it might be useful
# [(m.start(0), m.end(0)) for m in re.finditer(pattern, string)]
    for match in re.finditer(pattern, str(seq.seq)):
        print(match.start(0) + 1)


texts: A python script to plot a joint probability mass function table

texts:
Consider this example

  Suppose that 3 balls are randomly selected from an urn containing 3
  red, 4 white, and 5 blue balls. If we let X and Y denote,
  respectively, the number of red and white balls chosen, then the joint
  probability mass function of \$X\$ and \$Y\$, \$p(i,j) = P\{X=i,Y=j\}\$, is given by

This code is to plot such a joint probability mass function table.

arr = np.empty((4,4), dtype='U30')
sample_space = []
red_balls = 3
white_balls = 4
blue_balls = 5
for i in range(4):
    for j in range(4):
        r = comb(red_balls, i)
        w = comb(white_balls, j)
        b = comb(blue_balls, 3-i-j)
        result = int(r*w*b)
        arr[i][j] = r'$\dfrac{{{}}}{{{}}}$'.format(result, 220)
        sample_space.append(result)
pd.DataFrame(arr)

And I got.

question

if I want to compute the marginal probability I have to add another empty array which is inefficient, is there a better way to do that?


texts:
I'm writing a program to process log files.  Currently, I have a script that will extract the desired information from each log and put it into a list - I have about nine different lists all containing different information.

Currently, I'm loading the file, extracting the desired information, and storing it in a list, finally I write the lists to a CSV.

The program roughly follows this outline.

#load data

#create lists
list1 = []
list2 = []
list3 = []
list4 = []
list5 = []

#extract information to populate lists.

#write CSV's
with open('output1.csv', 'wb') as csvfile:
        writer = csv.writer(out)
    for line in list1:
        writer.writerow(line)

with open('output2.csv', 'wb') as csvfile:
    writer = csv.writer(out)
    for line in list2:
        writer.writerow(line)

with open('output3.csv', 'wb') as csvfile:
    writer = csv.writer(out)
    for line in list3:
        writer.writerow(line)

with open('output4.csv', 'wb') as csvfile:
    writer = csv.writer(out)
    for line in list4:
        writer.writerow(line)

with open('output5.csv', 'wb') as csvfile:
    writer = csv.writer(out)
    for line in list5:
        writer.writerow(line)

with open('output6.csv', 'wb') as csvfile:
    writer = csv.writer(out)
    for line in list6:
        writer.writerow(line)

I'm wondering if there is a better/faster way to do this.  Considering these lists can get pretty large. I feel like it might be quicker/faster to load the data and immediately write is to a file as it gets processed.  Thoughts?


texts: Four-lines script to make an immediate DB and document root zip backup

texts:
What do you think of my script to make an immediate DB and document root zip backup? Any flaw you might find. Please inform me of it:

#!/bin/bash
mysqldump -u root -p --all-databases | zip ${drt}/db-$(date +\%F-\%T).zip
zip -r all_zipped-$(date +\%F-\%T).zip ${drt}/ -x "*/cache/*"
rm -rf  ${drt}/db-$(date +\%F-\%T).zip

It is important to me to have the backup in document root because of comfortability. It's a small server environment (with external automatic backups) and I think it'll be redundant to have a backup dir just for once-in-a-while immediate backups I do with that nice script above.


texts:
I am pretty new to competitive programming. It took me almost 3 hours to make this, initially I started with 4 for loops and then to make it scalable for sizes bigger than 4, I decided to turn this into a recursion which is when it took most of my time, even though I am proud of it, I am still worried that it took me an hour to actually come up with the solution and I have a feeling that it is neither efficient nor good, (you could say I am basically doing combination rather than permutation),
I would like a code review please
and how would you make this more efficient?

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <string.h>

int counter = 0;

int factorial(int n){
    if(n==1) return 1;
    else return n*factorial(n-1);
}

int index_present_check(int* n_arr, int n, int depth){
    for(int i = 0 ; i < depth ; i++) if(n_arr[i]==n) return 1;
    return 0;
}

int present_in_array_check(char** store_arr,char* arr_to_print,int counter){
    for(int i = 0; i < counter; i++) if(strcmp(store_arr[i],arr_to_print)==0) return 1;
    return 0;
}

void permutation(int depth,int length,char* src_arr, char* arr_to_print,int*     n_arr,char** store_arr){
    for(int i = 0 ; i < length ; i ++){
        if(depth ==1){
            arr_to_print[0] = src_arr[i];
            n_arr[0] = i;
            permutation(depth+1,length,src_arr,arr_to_print,n_arr,store_arr);
        }
        else{
            if(!index_present_check(n_arr,i,depth)){
                arr_to_print[depth-1] = src_arr[i];
                n_arr[depth-1] = i;
                permutation(depth+1,length,src_arr,arr_to_print,n_arr,store_arr);
                n_arr[depth-1] = -1;
            }
        }
    }
    if(depth==length){
        if(!present_in_array_check(store_arr,arr_to_print,counter)){
            printf("%s ",arr_to_print);
            strcpy(store_arr[counter],arr_to_print);
            counter++;
            if(counter%factorial(length-1)==0){
                printf("\n");
            }
        }
    }
}

int main()
{
    char* src_arr = "ABCDE";
    int length = 4;
    int depth = 1;

    int iteration_length = factorial(length);

    char* arr_to_print = malloc(length*sizeof(char));
    int* n_arr = malloc(length*sizeof(int));
    char** store_arr = (char**)malloc(sizeof(char*)*iteration_length);
    for(int i = 0 ; i<iteration_length;i++) store_arr[i] = malloc(sizeof(char)*length);

    permutation(depth,length,src_arr,arr_to_print,n_arr,store_arr);

    return 0;
}

I posted this in reddit too, but it was a bit off timing and didn't get much code review, however it was helpful nonetheless.


texts:
I have simple code, that draw graph of my histogram. But code looks ugly, do you have idea, how can I make it looks better ?

            for (int i = 0; i < hueCount - 2; ++i)
            {
                if( i % 6 == 0)
                g.DrawLine(borderPen,
                   margin+ (width - 2 * margin) * ((float)i / (float)hueCount),
                   height - margin,
                   margin+ (width - 2 * margin) * ((float)i / (float)hueCount),
                   height - margin - 5);

                g.DrawLine(redPen,
                    margin + ((float)i / (float)hueCount) * (width - 2 * margin),
                    (-(float)reds[i] / (float)maxRed) * (height - 2 * margin) - margin + height,
                    margin + ((float)(i + 1) / (float)hueCount) * (width - 2 * margin),
                    (-(float)reds[i + 1] / (float)maxRed) * (height - 2 * margin) - margin + height);

                g.DrawLine(greenPen,
                    margin + ((float)i / (float)hueCount) * (width - 2 * margin),
                    (-(float)greens[i] / (float)maxGreen) * (height - 2 * margin) - margin + height,
                    margin + ((float)(i + 1) / (float)hueCount) * (width - 2 * margin),
                    (-(float)greens[i + 1] / (float)maxGreen) * (height - 2 * margin) - margin + height);

                g.DrawLine(bluePen,
                    margin + ((float)i / (float)hueCount) * (width - 2 * margin),
                    (-(float)blues[i] / (float)maxBlue) * (height - 2 * margin) - margin + height,
                    margin + ((float)(i + 1) / (float)hueCount) * (width - 2 * margin),
                    (-(float)blues[i + 1] / (float)maxBlue) * (height - 2 * margin) - margin + height);
            }


texts:
I frequently encounter the need for creating services that concurrently process messages from Redis queues. So I decided to share the knowledge I learned and help others easily bootstrap such a service. I would like a second opinion from code reviews of the library. Please do contribute if you feel like improving it!

Github repo: https://github.com/ikhoury/rstreamer

Sample usage

You create Worker classes that processes incoming messages from a queue.

public class SampleWorker implements Worker {

    @Override
    public void processSingleItem(String item) {
        System.out.println("Got item: " + item);
    }
}

You link workers to a task queue using a WorkSubscription:

List<Worker> workers = new ArrayList();
workers.add(new SampleWorker());
WorkSubscription subscription = new WorkSubscription("my:task:queue", workers);

You add your WorkSubscription to a SubscriptionManager that handles the lifecycle of all subscriptions.

subscriptionManager.addSubscription(subscription);
subscriptionManager.activateSubscriptions();

Background threads will spin up, continuously poll tasks, and process them using workers.