I'm facing an issue with allocating huge arrays in numpy on Ubuntu 18 while not facing the same issue on MacOS.
I am trying to allocate memory for a numpy array with... moreI'm facing an issue with allocating huge arrays in numpy on Ubuntu 18 while not facing the same issue on MacOS.
I am trying to allocate memory for a numpy array with shape (156816, 36, 53806) with
np.zeros((156816, 36, 53806), dtype='uint8')
and while I'm getting an error on Ubuntu OS
>>> import numpy as np
>>> np.zeros((156816, 36, 53806), dtype='uint8')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
numpy.core._exceptions.MemoryError: Unable to allocate array with shape (156816, 36, 53806) and data type uint8
I'm not getting it on MacOS:
>>> import numpy as np
>>> np.zeros((156816, 36, 53806), dtype='uint8')
array(,
,
,
...,
,
,
,
I am training on 970 samples and validating on 243 samples.
How big should batch size and number of epochs be when fitting a model in Keras to optimize the val_acc? Is there any... moreI am training on 970 samples and validating on 243 samples.
How big should batch size and number of epochs be when fitting a model in Keras to optimize the val_acc? Is there any sort of rule of thumb to use based on data input size?
I have a tensor with dimensions (30, 35, 49). I want to reshape it to (30, 35, 512) in order to be able to multiply with another tensor which has also the shape (30, 35,... moreI have a tensor with dimensions (30, 35, 49). I want to reshape it to (30, 35, 512) in order to be able to multiply with another tensor which has also the shape (30, 35, 512).
I want to do padding on the tensor with (30, 35, 49) dimension in order to make it (30, 35, 512) dimensional.
How can this be done?
First of all, I'm a beginner studying AI and this is not an opinion-oriented question or one to compare programming languages. I'm not implying that Python is the best language.... moreFirst of all, I'm a beginner studying AI and this is not an opinion-oriented question or one to compare programming languages. I'm not implying that Python is the best language. But the fact is that most of the famous AI frameworks have primary support for Python. They can even be multi language supported, for example, TensorFlow that supports Python, C++, or CNTK from Microsoft that support C# and C++, but the most used is Python (I mean more documentation, examples, bigger community, support, etc). Even if you choose C# (developed by Microsoft and my primary programming language) you must have the Python environment set up.
I read in other forums that Python is preferred for AI because the code is simplified and cleaner, good for fast prototyping.
So what is the big deal with Python?
Why is there a growing association between Python and AI? less
I want to get a list of the column headers from a pandas DataFrame. The DataFrame will come from user input so I won't know how many columns there will be or what they will be... moreI want to get a list of the column headers from a pandas DataFrame. The DataFrame will come from user input so I won't know how many columns there will be or what they will be called.
For example, if I'm given a DataFrame like this:
>>> my_dataframe
y gdp cap
0 1 2 5
1 2 3 9
2 8 7 2
3 3 4 7
4 6 7 7
5 4 8 3
6 8 2 8
7 9 9 10
8 6 6 4
9 10 10 7
I would get a list like this:
>>> header_list
I am totally new to Machine Learning and I have been working with unsupervised learning technique.
Image shows my sample Data(After all Cleaning) Screenshot : Sample Data
I have... moreI am totally new to Machine Learning and I have been working with unsupervised learning technique.
Image shows my sample Data(After all Cleaning) Screenshot : Sample Data
I have this two Pipline built to Clean the Data:
num_attribs = list(housing_num)
cat_attribs =
print(type(num_attribs))
num_pipeline = Pipeline()
cat_pipeline = Pipeline()
Then I did the union of this two pipelines and the code for the same is shown below :
from sklearn.pipeline import FeatureUnion
full_pipeline = FeatureUnion(transformer_list=)
Now I am trying to do fit_transform on the Data But Its showing Me the Error.
Code for Transformation:
housing_prepared = full_pipeline.fit_transform(housing)
housing_prepared
Error message:
fit_transform() takes 2 positional arguments but 3 were given
Im trying to write NN from scratch to better understand what is going on underneath Keras API. And now I have got problem with applying gradinets to my loss. I believe there is some problem with architecture and my understanding of TF. Basically line with grads returns Nones and hence code returns:
ValueError: No gradients provided for any variable: .
Here is my code. Its very straightforward but I want to do it simply before warping it into model class.
input_shape = x_train.shape
n_hidden_1 = 32
n_hidden_2 = 8
output_shape = 1
epochs = 1
How do I save a trained Naive Bayes classifier to disk and use it... moreHow do I save a trained Naive Bayes classifier to disk and use it to predict data?
I have the following sample program from the scikit-learn website:
from sklearn import datasets
iris = datasets.load_iris()
from sklearn.naive_bayes import GaussianNB
gnb = GaussianNB()
y_pred = gnb.fit(iris.data, iris.target).predict(iris.data)
print "Number of mislabeled points : %d" % (iris.target != y_pred).sum()
I ran a logistic regression model and made predictions of the logit values. I used this to get the points on the ROC curve:
from sklearn import metrics
fpr, tpr, thresholds =... moreI ran a logistic regression model and made predictions of the logit values. I used this to get the points on the ROC curve:
from sklearn import metrics
fpr, tpr, thresholds = metrics.roc_curve(Y_test,p)
I know metrics.roc_auc_score gives the area under the ROC curve. Can anyone tell me what command will find the optimal cut-off point (threshold value)?
I am using hadoop-1.2.1 and sqoop version is 1.4.4.
I am trying to run the following query.
sqoop import --connect jdbc:mysql://IP:3306/database_name --table clients --target-dir... moreI am using hadoop-1.2.1 and sqoop version is 1.4.4.
I am trying to run the following query.
sqoop import --connect jdbc:mysql://IP:3306/database_name --table clients --target-dir /data/clients --username root --password-file /sqoop.password -m 1
sqoop.password is a file which is kept on HDFS in path /sqoop.password with permission 400.
It is giving me an error
Access denied for user 'root'@'IP' (using password: YES)
Can anyone provide solution for this?
How do I delete all data from solr by command? We are using solr with lily and hbase.
How can I delete data from both hbase and... moreHow do I delete all data from solr by command? We are using solr with lily and hbase.
How can I delete data from both hbase and solr?
http://lucene.apache.org/solr/4_10_0/tutorial.html#Deleting+Data
Can I import RDBMS table data (table doesn't have a primary key) to hive using sqoop? If yes, then can you please give the sqoop import command.
I have tried with sqoop import... moreCan I import RDBMS table data (table doesn't have a primary key) to hive using sqoop? If yes, then can you please give the sqoop import command.
I have tried with sqoop import general command, but it failed.
I am trying to use Kafka.All configurations are done properly but when I try to produce message from console I keep getting the following error
WARN Error while fetching metadata... moreI am trying to use Kafka.All configurations are done properly but when I try to produce message from console I keep getting the following error
WARN Error while fetching metadata with correlation id 39 :
{4-3-16-topic1=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
Kafka version: 2.11-0.9.0.0
I exploring a few options to setup kafka and I knew that the Zookeeper has to be up and running to initiate a kafka.
I would like to know how can I find the below.
1) hostname and... moreI exploring a few options to setup kafka and I knew that the Zookeeper has to be up and running to initiate a kafka.
I would like to know how can I find the below.
1) hostname and port for my zookeeper instance---I checked the zoo.cfg and I could only find the ClientPort not the hostname, will hostname be the hostname of my box??
2) To check if ZooKeeper is up and running---I tried to do a ps -ef | grep "zoo" I could not find anything. May be I am using a wrong key word to search??
Any help would be really appreciated? less
How to copy file from HDFS to the local file system . There is no physical location of a file under the file , not even directory . how can i moved them to my local for further... moreHow to copy file from HDFS to the local file system . There is no physical location of a file under the file , not even directory . how can i moved them to my local for further validations.i am tried through winscp
If data reaching millions time computing is large.I want to compute in real time that i don't want to use Mapreduce
How to quickly count number of rows.
So far I have been unable to find an R library that can create a sunburst plot like those by John Stasko. Anyone knows how to accomplish that in R or Python?
Link shows a graphic visualization taken form census website. Link for the same is shared below. I want to create graphic visualization of the same kind in my python program.
Link... moreLink shows a graphic visualization taken form census website. Link for the same is shared below. I want to create graphic visualization of the same kind in my python program.
Link for the graphic visualization: http://www.census.gov/dataviz/visualizations/stem/stem-html/
Which kind of visualization is this? is it a piechart or any other different kind of graphic visualization.
Are there any APIs available to create such a visualization or do i need to use any web service?
I have created an array :
import numpy as np
data = np.zeros( (512,512,3), dtype=np.uint8)
data =
What I want this to do is display a single red dot in the center of a 512x512... moreI have created an array :
import numpy as np
data = np.zeros( (512,512,3), dtype=np.uint8)
data =
What I want this to do is display a single red dot in the center of a 512x512 image. (At least to begin with... I think I can figure out the rest from there)
I have a LDA model with the 10 most common topics in 10K documents. Now it's just an overview of the words with corresponding probability distribution for each topic.
I was... moreI have a LDA model with the 10 most common topics in 10K documents. Now it's just an overview of the words with corresponding probability distribution for each topic.
I was wondering if there is something available for python to visualize these topics?
I have a data set with huge number of features, so analysing the correlation matrix has become very difficult. I want to plot a correlation matrix which we get using... moreI have a data set with huge number of features, so analysing the correlation matrix has become very difficult. I want to plot a correlation matrix which we get using dataframe.corr() function from pandas library. Is there any built-in function provided by the pandas library to plot this matrix?
I am trying to use IPython notebook on MacOS X with Python 2.7.2 and IPython 1.1.0.I cannot get matplotlib graphics to show up inline.
import matplotlib
import numpy as np
import... moreI am trying to use IPython notebook on MacOS X with Python 2.7.2 and IPython 1.1.0.I cannot get matplotlib graphics to show up inline.
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
I have also tried %pylab inline and the ipython command line arguments --pylab=inline but this makes no difference.
x = np.linspace(0, 3*np.pi, 500)
plt.plot(x, np.sin(x**2))
plt.title('A simple chirp')
plt.show()
Instead of inline graphics, I get this:
<matplotlib.figure.Figure at 0x110b9c450>
And matplotlib.get_backend() shows that I have the 'module://IPython.kernel.zmq.pylab.backend_inline' backend. less