I'm trying to build a Python Lambda to send images to TensorFlow Serving for inferences. I have at least two dependencies: CV2 and tensorflow_serving.apis. I've run multiple... moreI'm trying to build a Python Lambda to send images to TensorFlow Serving for inferences. I have at least two dependencies: CV2 and tensorflow_serving.apis. I've run multiple tutorials showing it's possible to run tensorflow in a lambda, but they provide the package to install and don't explain how they got it to fit in the limit of less than 256MB unzipped.
How to Deploy ... Lambda and TensorFlow
Using TensorFlow and the Serverless Framework...
I've tried following the official instructions for packaging but just this downloads 475MB of dependencies:
$ python -m pip install tensorflow-serving-api --target .
Collecting tensorflow-serving-api
Downloading https://files.pythonhosted.org/packages/79/69/1e724c0d98f12b12f9ad583a3df7750e14ec5f06069aa4be8d75a2ab9bb8/tensorflow_serving_api-1.12.0-py2.py3-none-any.whl
...
$ du -hs .
475M .
I see that others have fought this dragon and won (1) (2) by doing contortions to rip out all unused libraries from all dependencies or compile from scratch. But... less
I began to fall in love with a Python Visualization library called Altair, and i use it with every small data science project that ive done.
Now, in terms of Industry use... moreI began to fall in love with a Python Visualization library called Altair, and i use it with every small data science project that ive done.
Now, in terms of Industry use cases, Does it make sense to visualize Big Data or should we just take a random sample?
I'm learning object oriented programing in a data science context.
I want to understand what good practice is in terms of writing methods within a class that relate to one... moreI'm learning object oriented programing in a data science context.
I want to understand what good practice is in terms of writing methods within a class that relate to one another.
When I run my code:
import pandas as pd
pd.options.mode.chained_assignment = None
class MyData:
def __init__(self, file_path):
self.file_path = file_path
def prepper_fun(self):
'''Reads in an excel sheet, gets rid of missing values and sets datatype to numerical'''
df = pd.read_excel(self.file_path)
df = df.dropna()
df = df.apply(pd.to_numeric)
self.df = df
return(df)
def quality_fun(self):
'''Checks if any value in any column is more than 10. If it is, the value is replaced with
a warning 'check the original data value'.'''
for col in self.df.columns:
for row in self.df.index:
if self.df > 10:
self.df = str('check original data value')
return(self.df)
so I happened to receive an xlms file that contains names of individuals with different titles such as Mr, Ms, Dr, Mrs, Judge etc. However some of these names contains multiple... moreso I happened to receive an xlms file that contains names of individuals with different titles such as Mr, Ms, Dr, Mrs, Judge etc. However some of these names contains multiple titles within one name example "Mr Mrs Ronderval", "Dr Rev Johns Mr" etc. So am trying to remove all of them except for one, hence the final result should be Mr Ronderval or Mrs Ronderval, Dr Johns or Rev Johns or Mr Johns any of them will be fine. So far what i have done is to convert the strings into a list of lists such as name_list = , and have a list of titles title=. So i tried to iterate through the name_list removing all values from titles and the result obviously is "Roderval" and "Johns" but i want atleast one title to be left in the name Mr Ronderval or Mrs Ronderval, Dr Johns or Rev Johns or Mr Johns. How do i go about this?
Here is my code using list comprehension
name_list=
I am facing problem while downloading 'caret' package in R studios. The code below was taken from the caret documentation.
install.packages("caret", dependencies = c("Depends",... moreI am facing problem while downloading 'caret' package in R studios. The code below was taken from the caret documentation.
install.packages("caret", dependencies = c("Depends", "Suggests"))
it works fine while installing but it gives Errors and Warnings while unpacking few packages like mentioned below:
ERROR: dependencies ‘eiPack’, ‘ei’, ‘MCMCpack’, ‘Zelig’ are not available for package ‘ZeligEI’
* removing ‘/home/shazil/R/x86_64-pc-linux-gnu-library/3.4/ZeligEI’
Warning in install.packages :
installation of package ‘ZeligEI’ had non-zero exit status
At the end when the whole installation process is finished it says:
The downloaded source packages are in
‘/tmp/RtmpeiP5GO/downloaded_packages’
After that when I use the library() command, the following Error appears
> library(caret)
Error in library(caret) : there is no package called ‘caret’
I am using Ubuntu 16.04, Dell machine Core i5 7th Gen, 6GB RAM AMD RADEON GRAPHICS
Would really appreciate... less
This is my problem: Cousera course on Apllied Data Science in Python I am doing Assigment 2.
Question 1 Which country has won the most gold medals in summer games? This function... moreThis is my problem: Cousera course on Apllied Data Science in Python I am doing Assigment 2.
Question 1 Which country has won the most gold medals in summer games? This function should return a single string value.
This my code:
def answer_one():
return df[df == df.index(0)
answer_one()
This is the error which I am getting:
NameError: name 'df' is not defined
Can any one tell my what that part (town = thisLine)exactly do?
def get_list_of_university_towns():
'''Returns a DataFrame of towns and the states they are in from the... moreCan any one tell my what that part (town = thisLine)exactly do?
def get_list_of_university_towns():
'''Returns a DataFrame of towns and the states they are in from the
university_towns.txt list. The format of the DataFrame should be:
DataFrame( [ , ,
columns= )
The following cleaning needs to be done:
1. For "State", removing characters from "[" to the end.
2. For "RegionName", when applicable, removing every character from " (" to the end.
3. Depending on how you read the data, you may need to remove newline character '\n'. '''
data =
state = None
state_towns =
with open('university_towns.txt') as file:
for line in file:
thisLine = line
if thisLine == '':
state = thisLine
continue
if '(' in line:
town = thisLine
state_towns.append()
else:
town = thisLine
state_towns.append()
data.append(thisLine)
df = pd.DataFrame(state_towns,columns = )
return df
I'm looking for information on how should a Python Machine Learning project be organized. For Python usual projects there is Cookiecutter and for R ProjectTemplate.
This is my... moreI'm looking for information on how should a Python Machine Learning project be organized. For Python usual projects there is Cookiecutter and for R ProjectTemplate.
This is my current folder structure, but I'm mixing Jupyter Notebooks with actual Python code and it does not seems very clear.
.
├── cache
├── data
├── my_module
├── logs
├── notebooks
├── scripts
├── snippets
└── tools
I work in the scripts folder and currently adding all the functions in files under my_module, but that leads to errors loading data(relative/absolute paths) and other problems.
I could not find proper best practices or good examples on this topic besides this kaggle competition solution and some Notebooks that have all the functions condensed at the start of such Notebook. less
I'm learning object oriented programing in a data science context.
I want to understand what good practice is in terms of writing methods within a class that relate to one... moreI'm learning object oriented programing in a data science context.
I want to understand what good practice is in terms of writing methods within a class that relate to one another.
When I run my code:
import pandas as pd
pd.options.mode.chained_assignment = None
class MyData:
def __init__(self, file_path):
self.file_path = file_path
def prepper_fun(self):
'''Reads in an excel sheet, gets rid of missing values and sets datatype to numerical'''
df = pd.read_excel(self.file_path)
df = df.dropna()
df = df.apply(pd.to_numeric)
self.df = df
return(df)
def quality_fun(self):
'''Checks if any value in any column is more than 10. If it is, the value is replaced with
a warning 'check the original data value'.'''
for col in self.df.columns:
for row in self.df.index:
if self.df > 10:
self.df = str('check original data value')
return(self.df)
I am relatively new Data science in python and was exploring some competition on data science, i am getting confused with "Training data Set" and "Test Data Set" . Some projects... moreI am relatively new Data science in python and was exploring some competition on data science, i am getting confused with "Training data Set" and "Test Data Set" . Some projects have merged both and some they have kept separate. What is the rationale behind having two data sets. Any advise will be helpful thanks
Can any one tell my what that part (town = thisLine)exactly do?
def get_list_of_university_towns():
'''Returns a DataFrame of towns and the states they are in from the... moreCan any one tell my what that part (town = thisLine)exactly do?
def get_list_of_university_towns():
'''Returns a DataFrame of towns and the states they are in from the
university_towns.txt list. The format of the DataFrame should be:
DataFrame( [ , ,
columns= )
The following cleaning needs to be done:
1. For "State", removing characters from "[" to the end.
2. For "RegionName", when applicable, removing every character from " (" to the end.
3. Depending on how you read the data, you may need to remove newline character '\n'. '''
data =
state = None
state_towns =
with open('university_towns.txt') as file:
for line in file:
thisLine = line
if thisLine == '':
state = thisLine
continue
if '(' in line:
town = thisLine
state_towns.append()
else:
town = thisLine
state_towns.append()
data.append(thisLine)
df = pd.DataFrame(state_towns,columns = )
return df
get_list_of_university_towns() less