I am working on a problem for a Intro to Data Science course on Coursera, and I am struggling with adding data to a column in a dataframe.
This is the data set I'm working... moreI am working on a problem for a Intro to Data Science course on Coursera, and I am struggling with adding data to a column in a dataframe.
This is the data set I'm working with:
SUMLEV REGION DIVISION STATE COUNTY STNAME CTYNAME
1 50 3 6 1 1 Alabama Autauga County
2 50 3 6 1 3 Alabama Baldwin County
3 50 3 6 1 5 Alabama Barbour County
4 50 3 6 1 7 Alabama Bibb County
What I am trying to do is to insert a column called TotalCounties that has the total count of counties by state as a last column. I've done similar things in SQL, but it doesn't seem to work quite the same in Python.
I have tried the code below, but the column ends up displaying as NaN instead of a number like I want it to.
counties_only_df = census_df[census_df
x = counties_only_df.groupby('STNAME').count()
counties_only_df = x
I have created 2 python sets created from 2 different CSV files which contains some stings.
I am trying to match the 2 sets so that it will return an intersection of the 2 (the... moreI have created 2 python sets created from 2 different CSV files which contains some stings.
I am trying to match the 2 sets so that it will return an intersection of the 2 (the common strings from both the sets should be returned).
This is how my code looks:
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
import string
import nltk
#using content mmanager to open and read file
#converted the text file into csv file at the source using Notepad++
with open(r'skills.csv', 'r', encoding="utf-8-sig") as f:
myskills = f.readlines()
#converting mall the string in the list to lowercase
list_of_myskills = map(lambda x: x.lower(), myskills)
set_of_myskills = set(list_of_myskills)
#print(type(nodup_filtered_content))
print(set_of_myskills)
#open and read by line from the text file
with open(r'list_of_skills.csv', 'r') as f2:
#using readlines() instead of read(), becasue it reads line by line (each
line as a string obj in the python list)
contents_f2 =... less
As someone who just got into data science (no prior coding history) I am new to using terminals, Python, and coding in general. While I do have some basic Python knowledge now,... moreAs someone who just got into data science (no prior coding history) I am new to using terminals, Python, and coding in general. While I do have some basic Python knowledge now, and I want to work on my first machine learning project, I am looking to use some packages that are not standard to python or jupyter lab, namely: TensorFlow.
After much struggle I was able to download TensorFlow in my terminal (i'm on Mac). Yet when I try to import to module I come to the following problem:
when I create a new file in jupyterlab (accessed via Anaconda) I have the option to create a python file using python 3 or python 3.7.2. When using python 3, I have access to packages to sklearn, SciPy, yet no TensorFlow. Then when I create a 3.7.2. file I can import the TensorFlow package, yet I cannot import the sklearn and SciPy packages anymore....
Did someone experience similar problems? Are there ways to solve this?
P.s. Using the 'pip install ...' command in terminal only sees to work rarely. Or I must be something... less
I am trying to do some deep learning work. For this, I first installed all the packages for deep learning in my Python environment.Here is what I did.In Anaconda, I created an... moreI am trying to do some deep learning work. For this, I first installed all the packages for deep learning in my Python environment.Here is what I did.In Anaconda, I created an environment called tensorflow as follows
conda create -n tensorflow
Then installed the data science Python packages, like Pandas, NumPy, etc., inside it. I also installed TensorFlow and Keras there. Here is the list of packages in that environment
(tensorflow) SFOM00618927A:dl i854319$ conda list
# packages in environment at /Users/i854319/anaconda/envs/tensorflow:
#
appdirs 1.4.3 <pip>
appnope 0.1.0 py36_0
beautifulsoup4 4.5.3 py36_0
bleach 1.5.0 py36_0
cycler 0.10.0 py36_0
decorator 4.0.11 py36_0
entrypoints 0.2.2 py36_1
freetype 2.5.5 ... less
Coming from a programming background where you write code, test, deploy, run.. I'm trying to wrap my head around the concept of "training a model" or a "trained model" in data... moreComing from a programming background where you write code, test, deploy, run.. I'm trying to wrap my head around the concept of "training a model" or a "trained model" in data science, and deploying that trained model.
I'm not really concerned about the deployment environment, automation, etc.. I'm trying to understand the deployment unit.. a trained model. What does a trained model look like on a file system, what does it contain?
I understand the concept of training a model, and splitting a set of data into a training set and testing set, but lets say I have a notebook (python / jupyter) and I load in some data, split between training/testing data, and run an algorithm to "train" my model. What is my deliverable under the hood? While I'm training a model I'd think there'd be a certain amount of data being stored in memory.. so how does that become part of the trained model? It obviously can't contain all the data used for training; so for instance if I'm training a chatbot agent (retrieval-based),... less
I have published a Tableau 9.3 viz on Tableau public: https://public.tableau.com/profile/michel.page#!/vizhome/exercice1/Courbesventesetprofit
I have succedded to have this viz... moreI have published a Tableau 9.3 viz on Tableau public: https://public.tableau.com/profile/michel.page#!/vizhome/exercice1/Courbesventesetprofit
I have succedded to have this viz displayed in a web page by integrating the code given by the 'Share' button on the Tableau public viz page.
Now I want to do the same, but inside an IPython notebook. It seems to be possible because I saw an example in nbviewer here: http://nbviewer.jupyter.org/gist/msund/96bd1d837f4139b2558d
I have integrated the 'Share' button script code into a Markdown cell but the viz won't get displayed when the cell is run. When I look at the browser console, it seems that the js code, and object tag get sanitized, even if I tell IPython to trust the notebook.
Is there any workaround ? less
Is there a way to work in Pycharm in local system (mac) which connect and execute the code via jupyter installed on Hadoop edge nodes ?
We have a requirement from data scientist... moreIs there a way to work in Pycharm in local system (mac) which connect and execute the code via jupyter installed on Hadoop edge nodes ?
We have a requirement from data scientist who would like to read data from HDFS using spark and plot some graphs or run some models in pycharm (mac) snippet by snippet.
Really interested to know if this model is possible or how generally data scientist work in big data eco system from local machine to read data from HDFS to run some models ?
I am trying to use IPython notebook on MacOS X with Python 2.7.2 and IPython 1.1.0.I cannot get matplotlib graphics to show up inline.
import matplotlib
import numpy as np
import... moreI am trying to use IPython notebook on MacOS X with Python 2.7.2 and IPython 1.1.0.I cannot get matplotlib graphics to show up inline.
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
I have also tried %pylab inline and the ipython command line arguments --pylab=inline but this makes no difference.
x = np.linspace(0, 3*np.pi, 500)
plt.plot(x, np.sin(x**2))
plt.title('A simple chirp')
plt.show()
Instead of inline graphics, I get this:
<matplotlib.figure.Figure at 0x110b9c450>
And matplotlib.get_backend() shows that I have the 'module://IPython.kernel.zmq.pylab.backend_inline' backend. less