I have a data matrix in "one-hot encoding" (all ones and zeros) with 260,000 rows and 35 columns. I am using Keras to train a simple neural network to predict a continuous... moreI have a data matrix in "one-hot encoding" (all ones and zeros) with 260,000 rows and 35 columns. I am using Keras to train a simple neural network to predict a continuous variable. The code to make the network is the following:
model = Sequential()
model.add(Dense(1024, input_shape=(n_train,)))
model.add(Activation('relu'))
model.add(Dropout(0.1))
sgd = SGD(lr=0.01, nesterov=True);
#rms = RMSprop()
#model.compile(loss='categorical_crossentropy', optimizer=rms, metrics=)
model.compile(loss='mean_absolute_error', optimizer=sgd)
model.fit(X_train, Y_train, batch_size=32, nb_epoch=3, verbose=1, validation_data=(X_test,Y_test), callbacks= )
However, during the training process, I see the loss decrease nicely, but during the middle of the second epoch, it goes to nan:
Train on 260000 samples, validate on 64905 samples
Epoch... less
While trying to validate the installation of tensorflow-gpu, I get an ImportError when trying to execute "import tensorflow as tf". I am using a Quadro K620 on Windows 7.... moreWhile trying to validate the installation of tensorflow-gpu, I get an ImportError when trying to execute "import tensorflow as tf". I am using a Quadro K620 on Windows 7. Tensorflow was installed using pip.
The following is the stack trace:
Microsoft Windows
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\aagarwal>python
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
Traceback (most recent call last):
File "C:\Users\aagarwal\AppData\Local\Programs\Python\Python35\lib\site-packag
es\tensorflow\python\pywrap_tensorflow_internal.py", line 18, in swig_import_hel
per
return importlib.import_module(mname)
File "C:\Users\aagarwal\AppData\Local\Programs\Python\Python35\lib\importlib\_
_init__.py", line 126, in import_module
return _bootstrap._gcd_import(name, package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File... less
Hi I just installed Tensorflow on my Mac and I want to use tf.contrib.slim but when I use it I get this
import tensorflow as tf
slim =... moreHi I just installed Tensorflow on my Mac and I want to use tf.contrib.slim but when I use it I get this
import tensorflow as tf
slim = tf.contrib.slim
Error:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/export/export_lib.py", line 25, in from tensorflow.python.saved_model.model_utils import build_all_signature_defs ModuleNotFoundError: No module named 'tensorflow.python.saved_model.model_utils'
I don't know what to do, I use Tensorflow.13.1 and python 3.7 less
gave this error in installation. Does this cause a problem?
ERROR: tensorboard 2.0.2 has requirement setuptools>=41.0.0, but you'll have setuptools 40.6.2 which is incompatible.
I'm saving my session state like so:
self._saver = tf.saver()
self._saver.save(self._session, '/network',... moreI'm saving my session state like so:
self._saver = tf.saver()
self._saver.save(self._session, '/network', global_step=self._time)
When I later restore I want to get the value of the global_step for the checkpoint I restore from. This is in order to set some hyper parameters from it.
The hacky way to do this would be to run through and parse the file names in the checkpoint directory. But surly there has to be a better, built in way to do this?
I have installed the Tensorflow bindings with python successfully. But when I try to import Tensorflow, I get the follwoing error.
ImportError: /lib/x86_64-linux-gnu/libc.so.6:... moreI have installed the Tensorflow bindings with python successfully. But when I try to import Tensorflow, I get the follwoing error.
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.17' not found (required by /usr/local/lib/python2.7/dist-packages/tensorflow/python/_pywrap_tensorflow.so)
I have tried to update GLIBC_2.15 to 2.17, but no luck.
I am working on training a VGG16-like model in Keras, on a 3 classes subset from Places205, and encountered the following error:
ValueError: Error when checking target: expected... moreI am working on training a VGG16-like model in Keras, on a 3 classes subset from Places205, and encountered the following error:
ValueError: Error when checking target: expected dense_3 to have shape (3,) but got array with shape (1,)
I read multiple similar issues but none helped me so far. The error is on the last layer, where I've put 3 because this is the number of classes I'm trying right now.
The code is the following:
import keras from keras.datasets
import cifar10 from keras.preprocessing.image
import ImageDataGenerator from keras.models
import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K import os
I am trying an Op that is not behaving as expected.
graph = tf.Graph()
with graph.as_default():
train_dataset = tf.placeholder(tf.int32,... moreI am trying an Op that is not behaving as expected.
graph = tf.Graph()
with graph.as_default():
train_dataset = tf.placeholder(tf.int32, shape=)
embeddings = tf.Variable(
tf.random_uniform(, -1.0, 1.0))
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
embed = tf.reduce_sum(embed, reduction_indices=0)
So I need to know the dimensions of the Tensor embed. I know that it can be done at the run time but it's too much work for such a simple operation. What's the easier way to do it?
Is there a way to plot both the training losses and validation losses on the same graph?
It's easy to have two separate scalar summaries for each of them individually, but this... moreIs there a way to plot both the training losses and validation losses on the same graph?
It's easy to have two separate scalar summaries for each of them individually, but this puts them on separate graphs. If both are displayed in the same graph it's much easier to see the gap between them and whether or not they have begin to diverge due to overfitting.
Is there a built in way to do this? If not, a work around way? Thank you much!
Perhaps too general a question, but can anyone explain what would cause a Convolutional Neural Network to diverge?Specifics:I am using Tensorflow's iris_training model with some... morePerhaps too general a question, but can anyone explain what would cause a Convolutional Neural Network to diverge?Specifics:I am using Tensorflow's iris_training model with some of my own data and keep gettingERROR:tensorflow: Model diverged with loss = NaN.Traceback...tensorflow.contrib.learn.python.learn.monitors.NanLossDuringTrainingError: NaN loss during training.Traceback originated with line:
tf.contrib.learn.DNNClassifier(feature_columns=feature_columns,
hidden_units=,
#optimizer=tf.train.ProximalAdagradOptimizer(learning_rate=0.001, l1_regularization_strength=0.00001),
n_classes=11,
model_dir="/tmp/iris_model")
I've tried adjusting the optimizer, using a zero for learning rate, and using no optimizer. Any insights into network layers, data size, etc is appreciated. less
I have been using the introductory example of matrix multiplication in TensorFlow.
matrix1 = tf.constant()
matrix2 = tf.constant(,)
product = tf.matmul(matrix1,... moreI have been using the introductory example of matrix multiplication in TensorFlow.
matrix1 = tf.constant()
matrix2 = tf.constant(,)
product = tf.matmul(matrix1, matrix2)
When I print the product, it is displaying it as a Tensor object:
<tensorflow.python.framework.ops.Tensor object at 0x10470fcd0>
But how do I know the value of product?The following doesn't help:
print product Tensor("MatMul:0", shape=TensorShape(), dtype=float32)
I know that graphs run on Sessions, but isn't there any way I can check the output of a Tensor object without running the graph in a session? less
When I run sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys}) I get InternalError: Blas SGEMM launch failed. Here is the full error and stack trace:... moreWhen I run sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys}) I get InternalError: Blas SGEMM launch failed. Here is the full error and stack trace:
InternalErrorTraceback (most recent call last)
<ipython-input-9-a3261a02bdce> in <module>()
1 batch_xs, batch_ys = mnist.train.next_batch(100)
----> 2 sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.pyc in run(self, fetches, feed_dict, options, run_metadata)
338 try:
339 result = self._run(None, fetches, feed_dict, options_ptr,
--> 340 run_metadata_ptr)
341 if run_metadata:
342 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.pyc in _run(self, handle, fetches, feed_dict, options, run_metadata)
562 try:
563 results = self._do_run(handle, target_list, unique_fetches,
--> 564 ... less
I found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss value.My... moreI found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss value.My questions are:1.Is there a more elegant or recommended way of regularization than doing it manually?2.I also find that get_variable has an argument regularizer. How should it be used? According to my observation, if we pass a regularizer to it (such as tf.contrib.layers.l2_regularizer, a tensor representing regularized term will be computed and added to a graph collection named tf.GraphKeys.REGULARIZATOIN_LOSSES. Will that collection be automatically used by TensorFlow (e.g. used by optimizers when training)? Or is it expected that I should use that collection by myself? less
I have trained a binary classification model with CNN, and here is my code
model = Sequential()
model.add(Convolution2D(nb_filters, kernel_size, kernel_size,
... moreI have trained a binary classification model with CNN, and here is my code
model = Sequential()
model.add(Convolution2D(nb_filters, kernel_size, kernel_size,
border_mode='valid',
input_shape=input_shape))
model.add(Activation('relu'))
model.add(Convolution2D(nb_filters, kernel_size, kernel_size))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
# (16, 16, 32)
model.add(Convolution2D(nb_filters*2, kernel_size, kernel_size))
model.add(Activation('relu'))
model.add(Convolution2D(nb_filters*2, kernel_size, kernel_size))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
# (8, 8, 64) = (2048)
model.add(Flatten())
model.add(Dense(1024))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(2)) # define a binary classification problem
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='adadelta',
metrics=)
model.fit(x_train,... less
tf.nn.embedding_lookup(params, ids, partition_strategy='mod', name=None)
I cannot understand the duty of this function. Is it like a lookup table? Which means to return the... moretf.nn.embedding_lookup(params, ids, partition_strategy='mod', name=None)
I cannot understand the duty of this function. Is it like a lookup table? Which means to return the parameters corresponding to each id (in ids)?For instance, in the skip-gram model if we use tf.nn.embedding_lookup(embeddings, train_inputs), then for each train_input it finds the correspond embedding?
It is a principal question, regarding the theory of neural networks:
Why do we have to normalize the input for a neural network?
I understand that sometimes, when for example the... moreIt is a principal question, regarding the theory of neural networks:
Why do we have to normalize the input for a neural network?
I understand that sometimes, when for example the input values are non-numerical a certain transformation must be performed, but when we have a numerical input? Why the numbers must be in a certain interval?
What will happen if the data is not normalized?
What is the difference between 'SAME' and 'VALID' padding in tf.nn.max_pool of tensorflow?In my opinion, 'VALID' means there will be no zero padding outside the edges when we do... moreWhat is the difference between 'SAME' and 'VALID' padding in tf.nn.max_pool of tensorflow?In my opinion, 'VALID' means there will be no zero padding outside the edges when we do max pool.According to A guide to convolution arithmetic for deep learning, it says that there will be no padding in pool operator, i.e. just use 'VALID' of tensorflow. But what is 'SAME' padding of max pool in tensorflow?
I am trying to visualise a transition from one state (work in origin country) and another (work in destination country). I think transition plot is perfect but I can't get how... moreI am trying to visualise a transition from one state (work in origin country) and another (work in destination country). I think transition plot is perfect but I can't get how does it work?
My data frame is called (Cimad) and my variables are EMPLOIPRE (origin) and EMPLOIFR (destination). Both are factors, the first count 20 levels, the second 15 levels (is it a problem, do I have to make them all match?) and I have 400 observations.
I hope you'll be able to help me because I am completely lost ! Thank you in advance less
I need to generate periodic (daily, monthly) web analytics dashboard reports. They will be static and don't require interaction, so imagine a PDF file as the target output. The... moreI need to generate periodic (daily, monthly) web analytics dashboard reports. They will be static and don't require interaction, so imagine a PDF file as the target output. The reports will mix tables and charts (mainly sparkline and bullet graphs created with ggplot2). Think Stephen Few/Perceptual Edge style dashboards, such as:
but applied to web analytics.
Any suggestions on what packages to use creating these dashboard reports?My first intuition is to use R markdown and knitr, but perhaps you've found a better solution. I can't seem to find rich examples of dashboards generated from R. less
Here are my codes, pretty standard but I am getting the error msg:
library(caret)
set.seed(32343)
modelFit = train(type~.,data=training,... moreHere are my codes, pretty standard but I am getting the error msg:
library(caret)
set.seed(32343)
modelFit = train(type~.,data=training, method='glm')
error msg:
Error in library(e1071) : there is no package called ‘e1071’
I would like to place two plots side by side using the ggplot2 package, i.e. do the equivalent of par(mfrow=c(1,2)).For example, I would like to have the following two plots show... moreI would like to place two plots side by side using the ggplot2 package, i.e. do the equivalent of par(mfrow=c(1,2)).For example, I would like to have the following two plots show side-by-side with the same scale.
x <- rnorm(100)
eps <- rnorm(100,0,.2)
qplot(x,3*x+eps)
qplot(x,2*x+eps)
Do I need to put them in the same data.frame?
qplot(displ, hwy, data=mpg, facets = . ~ year) + geom_smooth()
am using TensorFlow to train a neural network. This is how I am initializing the GradientDescentOptimizer:
init = tf.initialize_all_variables()
sess = tf.Session()... moream using TensorFlow to train a neural network. This is how I am initializing the GradientDescentOptimizer:
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
mse = tf.reduce_mean(tf.square(out - out_))
train_step = tf.train.GradientDescentOptimizer(0.3).minimize(mse)
The thing here is that I don't know how to set an update rule for the learning rate or a decay value for that.
How can I use an adaptive learning rate here?
I've been exploring the xgboost package in R and went through several demos as well as tutorials but this still confuses me: after using xgb.cv to do cross validation, how does... moreI've been exploring the xgboost package in R and went through several demos as well as tutorials but this still confuses me: after using xgb.cv to do cross validation, how does the optimal parameters get passed to xgb.train? Or should I calculate the ideal parameters (such as nround, max.depth) based on the output of xgb.cv?
param <- list("objective" = "multi:softprob",
"eval_metric" = "mlogloss",
"num_class" = 12)
cv.nround <- 11
cv.nfold <- 5
mdcv <-xgb.cv(data=dtrain,params = param,nthread=6,nfold = cv.nfold,nrounds = cv.nround,verbose = T)
md <-xgb.train(data=dtrain,params = param,nround = 80,watchlist = list(train=dtrain,test=dtest),nthread=6) less
When I trained my neural network with Theano or Tensorflow, they will report a variable called "loss" per epoch.
How should I interpret this variable? Higher loss is better or... moreWhen I trained my neural network with Theano or Tensorflow, they will report a variable called "loss" per epoch.
How should I interpret this variable? Higher loss is better or worse, or what does it mean for the final performance (accuracy) of my neural network?